Unmanned vehicle control and sonar operation in a marine environment

ABSTRACT

Many different types of systems are utilized or tasks are performed in a marine environment. The present invention provides various configurations of unmanned vehicles, or drones, that can be operated and/or controlled for such systems or tasks. One or more unmanned vehicles can be integrated with a dedicated marine electronic device of a marine vessel for autonomous control and operation. Additionally or alternatively, the unmanned vehicle can be manually remote operated during use in the marine environment. Such unmanned vehicles can be utilized in many different marine environment systems or tasks, including, for example, navigation, sonar, radar, search and rescue, video streaming, alert functionality, among many others. However, as contemplated by the present invention, the marine environment provides many unique challenges that may be accounted for with operation and control of an unmanned vehicle.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to and is a continuation of U.S. patentapplication Ser. No. 16/001,384 , entitled “Unmanned Vehicle Control andOperation in a Marine Environment”, filed Jun. 6, 2018, which is acontinuation of U.S. patent application Ser. No. 15/292,704, entitled“Unmanned Vehicle Control and Operation in a Marine Environment”, filedOct. 13, 2016, issued on Jul. 10, 2018 as U.S. Pat. No. 10,019,002, thecontents of each being incorporated by reference herein in theirentireties.

FIELD OF THE INVENTION

Embodiments of the present invention relate generally to unmannedvehicles (drones) and, more particularly, to systems, assemblies, andassociated methods for controlling and operating unmanned vehicles in amarine environment.

BACKGROUND OF THE INVENTION

Whether for recreation use, commercial use, or otherwise, persons in amarine environment utilize many different types of systems and performmany different types of tasks. There is always a need to improve suchsystems and offer more efficient ways for a user to enjoy the marineenvironment. With increased availability of unmanned vehicles (e.g.,drones), the present invention contemplates methods and systems detailedherein to utilize unmanned vehicles for various systems and tasks in themarine environment.

BRIEF SUMMARY OF THE INVENTION

The marine environment offers many unique circumstances and challengeswhen utilizing an unmanned vehicle. Embodiments of the present inventionprovide many different configurations and uses for an unmanned vehiclein a marine environment that account for these circumstances andchallenges. For example, using a marine electronic device from a marinevessel to autonomously control an unmanned vehicle can provide increasedcapability for unmanned vehicles in the marine environment. In thisregard, as detailed herein, embodiments of the present inventioncontemplate autonomous and manual remote control and operation ofunmanned vehicles in a marine environment for many different types oftasks and systems, including, for example, navigation, tracking, alertfunctionality, sonar, radar, search and rescue, video streaming, tasks,among many others.

An example embodiment of the present invention provides an unmannedvehicle for a marine environment. The unmanned vehicle is controllableby a marine electronic device of a marine vessel. The unmanned vehicleincludes a location sensor configured to gather location datacorresponding to the unmanned vehicle, a propulsion system configured topropel the unmanned vehicle, and at least one operational componentconfigured to gather operational data. The operational componentcomprises at least one of a sensor configured to obtain sensor data or acamera configured to obtained camera data. The unmanned vehicle furtherincludes a transmitter configured to transmit data to the marineelectronic device, a receiver configured to receive instructions fromthe marine electronic device, a processor, and a memory includingcomputer program product stored thereon. The computer program product isconfigured, when executed by the processor, to determine location dataindicative of a current location of the unmanned vehicle and transmitthe location data to the marine electronic device. The computer programproduct is further configured to receive a control signal from themarine electronic device. The control signal includes instructions totravel to a desired location based on a planned route or waypoint storedin memory of the marine electronic device. The computer program productis further configured to cause the propulsion system to propel theunmanned vehicle to the desired location, receive operational data fromthe at least one operational component, wherein the operational datacomprises at least one of sensor data or camera data, and transmit theoperational data to the marine electronic device.

In some embodiments, the computer program product is further configured,when executed by the processor, to receive a series of control signalsfrom the marine electronic device. Each control signal includesinstructions to travel to a new desired location so as to cause theunmanned vehicle to autonomously follow the planned route stored in thememory of the marine electronic device.

In some embodiments, the computer program product is further configured,when executed by the processor, to determine a location corresponding toone of a current location or a future location of the marine vessel andcause the propulsion system to cause the unmanned vehicle to travel tothe location corresponding to the one of the current location or thefuture location of the marine vessel such that the unmanned vehiclereturns to the marine vessel.

In some embodiments, the at least one sensor is a sonar systemconfigured to collect sonar data from an underwater environment. Thecomputer program product is further configured, when executed by theprocessor, to receive a control signal from the marine electronic devicewith instructions to gather sonar data, cause the sonar system to gathersonar data, determine location data associated with the gathered sonardata, and transmit the gathered sonar data and the location dataassociated with the gathered sonar data to the marine electronic devicefor display of the sonar data on a screen of the marine electronicdevice and association of the location of the unmanned vehicle with thegathered sonar data.

In some embodiments, the computer program product may be furtherconfigured, when executed by the processor, to cause the propulsionsystem to land the unmanned vehicle on a surface of a body of water suchthat at least a portion of the sonar system is submerged so as to gathersonar data from the underwater environment.

In some embodiments, the sonar system is deployable and the computerprogram product is further configured, when executed by the processor,to cause the propulsion system to cause the unmanned vehicle to hover ata hover position above a surface of a body of water and cause the sonarsystem to deploy from the hover position to beneath the surface of thebody of water such that at least a portion of the sonar system issubmerged so as to gather sonar data from the underwater environment.

In some embodiments, the computer program product is further configured,when executed by the processor, to receive a control signal from themarine electronic device with instructions to gather streaming video,cause the camera to gather camera data in the form of streaming video,determine location data associated with the gathered camera data, andtransmit the gathered camera data and the location data associated withthe gathered camera data to the marine electronic device for display ofthe streaming video on a screen of the marine electronic device andassociation of the location of the unmanned vehicle with the gatheredcamera data.

In some embodiments, the computer program product is further configured,when executed by the processor, to receive a series of control signalsfrom the marine electronic device. Each control signal may includeinstructions to travel to a new desired location such that the unmannedvehicle follows the marine vessel. The computer program product may befurther configured to cause the propulsion system to cause the unmannedvehicle to travel according to the series of control signals such thatthe unmanned vehicle follows the marine vessel and the camera datacaptures streaming video of the marine vessel.

In some embodiments, the at least one sensor is a wind sensor configuredto gather wind data including at least wind speed and wind direction.The computer program product is further configured, when executed by theprocessor, to receive a control signal from the marine electronic devicewith instructions to gather wind data, cause the wind sensor to gatherwind data, determine location data associated with the gathered winddata, and transmit the gathered wind data and the location dataassociated with the gathered wind data to the marine electronic devicefor association of the location of the unmanned vehicle with thegathered wind data.

In some embodiments, the computer program product is further configured,when executed by the processor, to receive a control signal from themarine electronic device with instructions to perform a search andrescue operation and cause the propulsion system to cause the unmannedvehicle to travel according to the search and rescue operation.

In some embodiments, the unmanned vehicle further includes deployableequipment. The deployable equipment may include at least one of a floatdevice, a rope, or a radio device. The computer program product isfurther configured, when executed by the processor, to cause thepropulsion system to cause the unmanned vehicle to hover at a hoverposition above a surface of a body of water at a location correspondingto a person in the water and cause deployment of the deployableequipment from the hover position to the surface of the body of watersuch that the person in the water has access to the deployableequipment.

In some embodiments, the unmanned vehicle further comprises at least onelight. The computer program product is further configured, when executedby the processor, to cause the propulsion system to cause the unmannedvehicle to hover at a hover position above a surface of a body of waterat a location corresponding to a person in the water and cause emissionof light at the location corresponding to the person in the water suchthat the light covers the person and visually marks the location of theperson in the water.

In some embodiments, the at least one sensor comprises at least aninfrared sensor configured to gather infrared data. The computer programproduct is further configured, when executed by the processor, to causeoperation of the infrared sensor as the unmanned vehicle travels togather infrared data to locate a person in a body of water, determinelocation data associated with the gathered infrared data, and transmitthe gathered infrared data and the location data associated with thegathered infrared data to the marine electronic device for display ofthe infrared data on a screen of the marine electronic device andassociation of the location of the unmanned vehicle with the gatheredinfrared data.

In some embodiments, the unmanned vehicle further comprises deployablebait or chum. The computer program product is further configured, whenexecuted by the processor, to cause the propulsion system to cause theunmanned vehicle to hover at a hover position above a surface of a bodyof water at the desired location and cause deployment of the deployablebait or chum from the hover position to the body of water.

In some embodiments, the unmanned vehicle is an unmanned aerial vehiclesuch that the propulsion system is configured to cause the unmannedaerial vehicle to fly around the marine environment.

In some embodiments, the unmanned vehicle is an unmanned surface vehiclesuch that the propulsion system is configured to cause the unmannedsurface vehicle to travel along a surface of a body of water of themarine environment.

In some embodiments, the unmanned vehicle is an unmanned underwatervehicle such that the propulsion system is configured to cause theunmanned underwater vehicle to travel beneath a surface of a body ofwater of the marine environment.

In some embodiments, the at least one sensor is at least one of a sonarsystem, a radar system, a microphone, a temperature sensor, a windsensor, a heave/roll sensor, or an IR sensor.

In some embodiments, the computer program product is further configured,when executed by the processor, to receive a landing control signal fromthe marine electronic device with instructions to land on the marinevessel while the marine vessel is travelling along a surface of a bodyof water. The landing control signal includes a location of the marinevessel, a direction of travel of the marine vessel, and a speed oftravel of the marine vessel. The computer program product may be furtherconfigured to cause the propulsion system to propel the unmannedvehicle, based on the location of the marine vessel, the speed of travelof the marine vessel, and the direction of travel of the marine vessel,to land on the marine vessel.

In some embodiments, the computer program product is further configured,when executed by the processor, to determine a second current locationand cause the propulsion system to cause the unmanned vehicle to returnto the current location in an instance in which the second currentlocation is at least a pre-determined distance from the current locationsuch that the unmanned vehicle is virtually anchored and configured tobe maintained at the current location.

In another example embodiment of the present invention a marineelectronic device of a marine vessel configured for control of anunmanned vehicle is provided. The marine electronic device comprises atransmitter configured to transmit instructions to the unmanned vehicle,a receiver configured to receive data from the unmanned vehicle, aprocessor, and a memory including computer program product storedthereon. The computer program product is configured, when executed bythe processor, to receive location data from the unmanned vehicle,determine, based on the received location data, the current location ofthe unmanned vehicle, determine, based on a planned route or waypointstored in the memory of the marine electronic device, a desired locationfor the unmanned vehicle, and transmit a control signal to the unmannedvehicle to instruct the unmanned vehicle to travel to the desiredlocation. The control signal causes the unmanned vehicle to travel tothe desired location. The computer program product is further configuredto receive operational data from the unmanned vehicle. The operationaldata comprises at least one of sensor data obtained by the unmannedvehicle or camera data obtained by the unmanned vehicle.

In some embodiments, the computer program product is further configured,when executed by the processor, to transmit a series of control signalsto the unmanned vehicle. Each control signal includes instructions tocause the unmanned vehicle to travel to a new desired location so as tocause the unmanned vehicle to autonomously follow the planned routestored in the memory of the marine electronic device.

In some embodiments, the computer program product is further configured,when executed by the processor, to cause display of the current locationof the unmanned vehicle on a screen of the marine electronic device.

In some embodiments, the at least one sensor of the unmanned vehicle isa sonar system configured to collect sonar data from an underwaterenvironment. The computer program product of the marine electronicdevice is further configured, when executed by the processor, totransmit a control signal to the unmanned vehicle with instructions togather sonar data, receive the gathered sonar data, and cause display ofthe gathered sonar data on a screen of the marine electronic device.

In some embodiments, the computer program product of the marineelectronic device is further configured, when executed by the processor,to receive location data associated with the gathered sonar data,determine a location associated with the gathered sonar data, store thegathered sonar data in association with the location in the memory, andcause display of an indication of the location associated with thegathered sonar data on the screen while causing display of the gatheredsonar data.

In some embodiments, the computer program product of the marineelectronic device is further configured, when executed by the processor,to transmit a control signal to the unmanned vehicle with instructionsto gather camera data in the form of streaming video, receive thegathered camera data, and cause display of the streaming video on ascreen of the marine electronic device.

In some embodiments, the computer program product of the marineelectronic device is further configured, when executed by the processor,to receive location data associated with the gathered camera data,determine a location associated with the gathered camera data, store thegathered camera data in association with the location in the memory, andcause display of an indication of the location associated with thegathered camera data on the screen while causing display of the gatheredcamera data.

In some embodiments, the computer program product of the marineelectronic device is further configured, when executed by the processor,to cause display of an image corresponding to the camera data over achart at the associated location.

In some embodiments, the computer program product is further configured,when executed by the processor, to transmit a series of control signalsto the unmanned vehicle. Each control signal includes instructions tocause the unmanned vehicle to travel to a new desired location such thatthe unmanned vehicle follows the marine vessel and the camera datacaptures streaming video of the marine vessel.

In some embodiments, the at least one sensor of the unmanned vehicle isa wind sensor configured to gather wind data including at least windspeed and wind direction. The computer program product of the marineelectronic device is further configured, when executed by the processor,to transmit a control signal to the unmanned vehicle with instructionsto gather wind data, receive the gathered wind data, receive locationdata associated with the gathered wind data, determine a locationassociated with the gathered wind data, and cause display of the winddata and an indication of the location associated with the gathered winddata on a screen of the marine electronic device.

In some embodiments, the computer program product is further configured,when executed by the processor, to transmit a control signal to theunmanned vehicle with instructions to cause the unmanned vehicle toperform a search and rescue operation.

In some embodiments, the unmanned vehicle includes deployable equipment.The deployable equipment includes at least one of a float device, arope, or a radio device. The computer program product of the marineelectronic device is further configured, when executed by the processor,to transmit a control signal to the unmanned vehicle to cause theunmanned vehicle to hover at a hover position above a surface of a bodyof water at a location corresponding to a person in the water and deploythe deployable equipment from the hover position to the surface of thebody of water such that the person in the water has access to thedeployable equipment.

In some embodiments, the unmanned vehicle includes at least one light.The computer program product of the marine electronic device is furtherconfigured, when executed by the processor, to transmit a control signalto the unmanned vehicle to cause the unmanned vehicle to hover at ahover position above a surface of a body of water at a locationcorresponding to a person in the water and cause emission of light atthe location corresponding to the person in the water such that thelight covers the person and visually marks the location of the person inthe water.

In some embodiments, the at least one sensor of the unmanned vehiclecomprises at least an infrared sensor configured to gather infrareddata. The computer program product of the marine electronic device isfurther configured, when executed by the processor, to transmit acontrol signal to the unmanned vehicle to cause operation of theinfrared sensor as the unmanned vehicle travels to gather infrared datato locate a person in a body of water, receive the gathered infrareddata and location data corresponding to the gathered infrared data, anddetermine, based on the location data and the gathered infrared data, alocation of the person in the body of water.

In some embodiments, the unmanned vehicle includes deployable bait orchum. The computer program product of the marine electronic device isfurther configured, when executed by the processor, to transmit acontrol signal to the unmanned vehicle to cause the unmanned vehicle tohover at a hover position above a surface of a body of water at alocation and deploy the deployable equipment from the hover position tothe body of water at the location.

In some embodiments, the computer program product is further configured,when executed by the processor, to transmit a landing control signal tothe unmanned vehicle with instructions to cause the unmanned vehicle toland on the marine vessel while the marine vessel is travelling along asurface of a body of water, wherein the landing control signal includesa location of the marine vessel, a direction of travel of the marinevessel, and a speed of travel of the marine vessel.

In yet another example embodiment of the present invention, a system foroperating an unmanned vehicle in a marine environment is provided. Thesystem comprises an unmanned vehicle including a location sensorconfigured to gather location data corresponding to the unmannedvehicle, a propulsion system configured to propel the unmanned vehicle,and at least one operational component configured to gather operationaldata. The operational component comprises at least one of a sensorconfigured to obtain sensor data or a camera configured to obtainedcamera data. The unmanned vehicle further includes a transmitterconfigured to transmit data to the marine electronic device, a receiverconfigured to receive instructions from the marine electronic device, anunmanned vehicle processor, and a memory including unmanned vehiclecomputer program product stored thereon. The system further comprises amarine electronic device of a marine vessel configured for control ofthe unmanned vehicle. The marine electronic device includes atransmitter configured to transmit instructions to the unmanned vehicle,a receiver configured to receive data from the unmanned vehicle, amarine electronic device processor, and a memory including marineelectronic device computer program product stored thereon. The unmannedvehicle computer program product is configured, when executed by theunmanned vehicle processor, to determine location data indicative of acurrent location of the unmanned vehicle, transmit the location data tothe marine electronic device, receive operational data from the at leastone operational component, wherein the operational data comprises atleast one of sensor data or camera data, and transmit the operationaldata to the marine electronic device. The marine electronic devicecomputer program product is configured, when executed by the marineelectronic device processor, to receive location data from the unmannedvehicle, determine, based on the received location data, the currentlocation of the unmanned vehicle, determine, based on a planned route orwaypoint stored in the memory of the marine electronic device, a desiredlocation for the unmanned vehicle, transmit a control signal to theunmanned vehicle to instruct the unmanned vehicle to travel to thedesired location, wherein the control signal causes the unmanned vehicleto travel to the desired location, and receive operational data from theunmanned vehicle, wherein the operational data comprises at least one ofsensor data obtained by the unmanned vehicle or camera data obtained bythe unmanned vehicle. The unmanned vehicle computer program product isfurther configured, when executed by the unmanned vehicle processor, toreceive a control signal from the marine electronic device, wherein thecontrol signal includes instructions to travel to a desired locationbased on a planned route or waypoint stored in memory of the marineelectronic device and cause the propulsion system to propel the unmannedvehicle to the desired location.

In some embodiments, the unmanned vehicle and/or marine electronicdevice of the system may be configured according to any features orconfigurations described herein, such as according to the abovedescribed example embodiments of an unmanned vehicle and/or marineelectronic device.

In other example embodiments, corresponding methods, apparatuses, andcomputer program products according to any features or configurationsdescribed herein may be provided.

BRIEF DESCRIPTION OF THE DRAWINGS

Having thus described the invention in general terms, reference will nowbe made to the accompanying drawings, which are not necessarily drawn toscale, and wherein:

FIG. 1 illustrates a marine environment that includes a watercraft andan unmanned vehicle, in accordance with some embodiments discussedherein;

FIG. 2A shows an example unmanned aerial vehicle, in accordance withsome embodiments discussed herein;

FIG. 2B shows an example unmanned surface vehicle, in accordance withsome embodiments discussed herein;

FIG. 2C shows an example unmanned submersible vehicle, in accordancewith some embodiments discussed herein;

FIG. 3A shows an example screen of a marine electronic device, whereinthe screen shows an example chart of a body of water withrepresentations of a watercraft and two unmanned vehicles, in accordancewith some embodiments discussed herein;

FIG. 3B shows an example screen of a marine electronic device, whereinthe screen shows an example chart of a body of water withrepresentations of a watercraft, an unmanned vehicle, and a path takenby the unmanned vehicle, in accordance with some embodiments discussedherein;

FIG. 3C shows an example screen of a marine electronic device, whereinthe screen shows an example chart of a body of water withrepresentations of a watercraft, two unmanned vehicles, andcorresponding survey areas for each unmanned vehicle, in accordance withsome embodiments discussed herein;

FIG. 4 illustrates an example marine environment with a watercraft andan unmanned vehicle, wherein the watercraft includes a landing site forthe unmanned vehicle, in accordance with some embodiments discussedherein;

FIG. 5 illustrates a schematic of a marine environment with a watercraftand an unmanned vehicle, wherein the watercraft includes a safe landingzone for the unmanned vehicle, in accordance with some embodimentsdiscussed herein;

FIG. 6 shows an example screen of a marine electronic device, wherein auser is drawing a safe landing zone for the unmanned vehicle, inaccordance with some embodiments discussed herein;

FIG. 7A shows an example screen of a marine electronic device, whereinthe screen shows video taken from the unmanned vehicle on a left portionof the screen and a chart illustrating locations of the watercraft andunmanned vehicle on a body of water on the right portion of the screen,in accordance with some embodiments discussed herein;

FIG. 7B shows an example screen of a marine electronic device, whereinthe screen shows video taken from the unmanned vehicle on a left portionof the screen and a chart illustrating locations of the watercraft andunmanned vehicle on a body of water on the right portion of the screen,wherein images from the video are overlaid on the chart, in accordancewith some embodiments discussed herein;

FIG. 8 illustrates a schematic of a marine environment with a watercraftand an unmanned vehicle, wherein the unmanned vehicle includes a camerathat is aimed toward the watercraft, in accordance with some embodimentsdiscussed herein;

FIG. 9A shows an example screen of a marine electronic device, whereinthe screen shows sonar imagery taken from the unmanned vehicle on a leftportion of the screen and a chart illustrating locations of thewatercraft and unmanned vehicle on a body of water on the right portionof the screen, in accordance with some embodiments discussed herein;

FIG. 9B shows an example screen of a marine electronic device, whereinthe screen shows sonar imagery taken from the unmanned vehicle on a leftportion of the screen and a chart illustrating locations of thewatercraft and unmanned vehicle on a body of water on the right portionof the screen, wherein the sonar imagery is overlaid on the chart, inaccordance with some embodiments discussed herein;

FIG. 10A illustrates an example marine environment with a watercraft andan unmanned vehicle, wherein the unmanned vehicle has landed on thewater surface and includes a transducer assembly that is emitting sonarbeams into the underwater environment, in accordance with someembodiments discussed herein;

FIG. 10B illustrates an example marine environment with a watercraft andan unmanned vehicle, wherein the unmanned vehicle is hovering above thewater surface and has deployed a transducer assembly into the water toemit sonar beams into the underwater environment, in accordance withsome embodiments discussed herein;

FIG. 11 shows an example screen of a marine electronic device, whereinthe screen shows radar imagery taken from the unmanned vehicle on a leftportion of the screen and a chart illustrating locations of thewatercraft and unmanned vehicle on a body of water on the right portionof the screen, in accordance with some embodiments discussed herein;

FIG. 12 illustrates an example marine environment with a watercraft andan unmanned vehicle, wherein the unmanned vehicle has landed on thewater surface, in accordance with some embodiments discussed herein;

FIG. 13A illustrates a schematic of a marine environment with anunmanned vehicle, wherein the unmanned vehicle includes a camera that isaimed toward a fish underneath the water surface, in accordance withsome embodiments discussed herein;

FIG. 13B shows an example screen of a marine electronic device, whereinthe screen shows video taken from the unmanned vehicle, and wherein auser has indicated an object for possible tracking, in accordance withsome embodiments discussed herein;

FIG. 13C shows an example screen of a marine electronic device, whereinthe screen shows radar imagery taken from the unmanned vehicle, andwherein a user has indicated an object for possible tracking, inaccordance with some embodiments discussed herein;

FIG. 13D shows an example screen of a marine electronic device, whereinthe screen shows a body of water with representation of the watercraft,the unmanned vehicle, and an object, in accordance with some embodimentsdiscussed herein;

FIG. 14 illustrates a schematic of a marine environment with twounmanned vehicles and an object, in accordance with some embodimentsdiscussed herein;

FIG. 15 illustrates a marine environment with an unmanned vehicle and aperson overboard, wherein the unmanned vehicle is emitting a light ontothe person, in accordance with some embodiments discussed herein;

FIG. 16 illustrates a marine environment with an unmanned vehicle and aperson overboard, wherein the unmanned vehicle has deployed a rope forthe person, in accordance with some embodiments discussed herein;

FIG. 17 shows an example screen of a marine electronic device, whereinthe screen shows a body of water with representation of a watercraft, anunmanned vehicle, and areas where wireless Internet is available, inaccordance with some embodiments discussed herein;

FIG. 18 illustrates an example marine environment with a watercraft andan unmanned vehicle, wherein the unmanned vehicle acts as a buoy tovisually indicate a location of a desired underwater feature, inaccordance with some embodiments discussed herein;

FIG. 19 shows a block diagram illustrating an example system forcontrolling and operating an unmanned vehicle, in accordance with someembodiments discussed herein;

FIG. 20 shows a block diagram illustrating an example environment forexample systems for controlling and operating multiple unmannedvehicles, in accordance with some embodiments discussed herein;

FIG. 21 illustrates a flow control diagram of an example method ofcontrolling and operating an unmanned vehicle to cause the unmannedvehicle to travel to a desired location, in accordance with someembodiments discussed herein; and

FIG. 22 illustrates a flow control diagram of an example method ofcontrolling and operating an unmanned vehicle to cause the unmannedvehicle to gather operational data, in accordance with some embodimentsdiscussed herein.

DETAILED DESCRIPTION

Exemplary embodiments of the present invention now will be describedmore fully hereinafter with reference to the accompanying drawings, inwhich some, but not all embodiments of the invention are shown. Indeed,the invention may be embodied in many different forms and should not beconstrued as limited to the exemplary embodiments set forth herein;rather, these embodiments are provided so that this disclosure willsatisfy applicable legal requirements. Like reference numerals refer tolike elements throughout.

Overview

Embodiments of the present invention contemplate many differentconfigurations and uses of unmanned vehicles (e.g., drones) in a marineenvironment. FIG. 1 shows a marine vessel 10 and an unmanned vehicle 20in a marine environment 1. As depicted, the marine vessel 10 is floatingon a surface 12 of a body of water 11 and an unmanned vehicle 20 isflying nearby. In such an embodiment, the unmanned vehicle 20 isconfigured for flight (i.e., an unmanned aerial vehicle (UAV)). In someembodiments, as will be described in greater detail herein, the unmannedvehicle 20 may be controlled and/or operated by or through a marineelectronic device 40 of the marine vessel 10.

Though the depicted embodiment of FIG. 1 shows an example marine vesselas a surface watercraft, other types of marine vessels are contemplatedby embodiments of the present invention described herein (e.g.,submersible marine vessels, hovercraft marine vessels, etc.).

Similarly, though the depicted embodiment of FIG. 1 shows an exampleunmanned vehicle as an unmanned aerial vehicle, other types of unmannedvehicles are contemplated by embodiments of the present inventiondescribed herein (e.g., unmanned surface vehicle (shown in FIG. 2B),unmanned submersible vehicle (shown in FIG. 2C, etc.). Further, thelabeling of “aerial”, “surface”, or “submersible” is not meant to belimiting, as many unmanned vehicles are capable of more than one type oftravel (e.g., aerial, surface, and submersible travel in a marineenvironment). The following description regarding FIGS. 2A, 2B, and 2Cdetail some components of unmanned vehicles that can be used inaccordance with example embodiments described herein. Indeed, when anexample embodiment describes an unmanned vehicle 20, functionality isnot meant to be limited to any one type of travel and the unmannedvehicle 20 is meant to include possible components as described withrespect to unmanned vehicle 120 in FIG. 19.

FIG. 2A illustrates a schematic representation of an example unmannedaerial vehicle (UAV) 20, which shows a possible example configuration ofan unmanned vehicle 120 described further respect to FIG. 19 (andcontemplated in embodiments of the present invention). The depictedunmanned aerial vehicle 20 defines a housing 22 that includes fourlanding elements 27 (though any number of landing elements arecontemplated) that enable the unmanned aerial vehicle 20 to land (suchas on land or a surface of the marine vessel). The unmanned aerialvehicle 20 also includes a propulsion system (e.g., propulsion system125 of FIG. 19) that includes four propellers 25 (though any number ortype of aerial-capable propulsion system is contemplated). Thepropulsion system of the unmanned aerial vehicle 20 is configured tocause the unmanned aerial vehicle 20 to fly according to its desireddirection and speed. In some embodiments, this includes, for example,the ability to hover above water, fly (or travel) to a desiredlocation/position, etc. Further, in some embodiments, the propulsionsystem of the unmanned aerial vehicle may be configured (through thepropellers 25 or other additional or different propulsion system) tocause the unmanned aerial vehicle 20 to submerge underwater and/ortravel on the surface of the water or land. In some embodiments, thehousing 22 may be configured to float on the surface of the water.

As depicted in FIG. 2A, the unmanned aerial vehicle 20 may include apayload 30 (or multiple payloads). The payload 30 may be any type ornumber of component(s), equipment, senor(s), or system(s) (e.g.,operational component(s) 130, lights 138, deployable equipment 190,location sensor 128, releasable attachment mechanism, etc.). Notably,while the payload 30 is shown near the bottom of the housing 22 of theunmanned aerial vehicle 22, any location, position, or orientation withrespect to the unmanned aerial vehicle 20 is contemplated. For example,in some embodiments described herein, the payload 30 includes a sonarsystem (e.g., sonar system 136) that is designed to be submerged whenthe unmanned aerial vehicle 20 floats on the surface of the water.Further, though not shown, the housing 22 of the unmanned aerial vehicle20 may be designed to have an accessible compartment and/or a releasableattachment mechanism such as for access by a user and/or for carryingitems that may be placed into the compartment or collected.

FIG. 2B illustrates a schematic representation of an example unmannedsurface vehicle 20′, which shows a possible example configuration of anunmanned vehicle 120 described further respect to FIG. 19 (andcontemplated in embodiments of the present invention). The depictedunmanned surface vehicle 20′ defines a housing 22′ that is designed tofloat on the surface 12 of the body of water 11. Additionally, theunmanned surface vehicle includes a propulsion system (e.g., propulsionsystem 125 of FIG. 19) that includes a motor 25′. The propulsion systemof the unmanned surface vehicle 20′ is configured to cause the unmannedsurface vehicle 20′ to travel along the surface of the body of wateraccording to a desired direction and speed. In some embodiments, thepropulsion system of the unmanned surface vehicle may be configured(through the motor 25′ or other additional or different propulsionsystem) to cause the unmanned surface vehicle 20′ to submergeunderwater, fly, and/or travel on the surface of the land. Like theunmanned aerial vehicle 20, the unmanned surface vehicle 20′ includes apayload 30′. In this regard, the payload 30′ and/or housing 22′ may beconfigured in any manner described herein in accordance with embodimentsof the present invention, such as described above with respect to theunmanned aerial vehicle 20.

FIG. 2C illustrates a schematic representation of an example unmannedsubmersible vehicle 20″, which shows a possible example configuration ofan unmanned vehicle 120 described further respect to FIG. 19 (andcontemplated in embodiments of the present invention). The depictedunmanned submersible vehicle 20″ defines a housing 22″ that is designedto be submerged under the surface 12 of the body of water 11.Additionally, the unmanned submersible vehicle includes a propulsionsystem (e.g., propulsion system 125 of FIG. 19) that includes a motor25″. The propulsion system of the unmanned submersible vehicle 20″ isconfigured to cause the unmanned submersible vehicle 20″ to travelunderneath the surface of the body of water according to a desireddirection and speed. In some embodiments, the propulsion system of theunmanned submersible vehicle may be configured (through the motor 25″ orother additional or different propulsion system) to cause the unmannedsubmersible vehicle 20′ to travel on the surface of the water or landand/or fly. Like the unmanned aerial vehicle 20, the unmannedsubmersible vehicle 20″ includes a payload 30″. In this regard, thepayload 30″ and/or housing 22″ may be configured in any manner describedherein in accordance with embodiments of the present invention, such asdescribed above with respect to the unmanned aerial vehicle 20.

The marine environment offers many different challenges or circumstancesthat are not contemplated during typical operation and control of anunmanned vehicle. One such challenges is that there is a moving base ofoperation (e.g., the marine vessel (where the unmanned vehicle isstored, starts from, lands, charges, receives it instructions, etc.) iscapable of moving). This is compounded when attempting to land to orientthe unmanned vehicle with respect to the orientation of the marinevessel (which is often constantly changing). Other challenges include,for example, changing tides that cause the surface of the water to riseor drop, weather issues, limited wireless range, limited battery life,limited safe landing conditions (in a typical environment an unmannedvehicle can land almost anywhere and the operator can simply go thereand pick it up), among many others.

In order to account for such challenges or circumstances and/or providedesired functionality in a marine environment, embodiments of thepresent invention contemplate many different configurations foroperation and control of an unmanned vehicle in a marine environment.Some such configurations are described herein (though this disclosure isnot meant to be limited to only the described examples, as manydifferent uses of an unmanned vehicle in a marine environment arecontemplated).

Integration with Marine Electronic Device

In some embodiments, the unmanned vehicle 120 (described further withrespect to FIG. 19) may be integrated with (e.g., controlled wirelesslyby) a marine electronic device 105 (described further with respect toFIG. 19) dedicated to a marine vessel. Such control can be accomplishedby a user by operating the marine electronic device 105 and/orautonomously by the marine electronic device 105 such as in accordancewith the functionality of the marine electronic device 105. Byintegrating the unmanned vehicle 120 with the marine electronic device105, many advantages can be realized. The functionally of the marineelectronic device 105, such as the navigation system (stored waypoints,routes, etc.), alert systems, control systems, processing, etc., can beutilized with or by the unmanned vehicle (and components thereof).Additionally, operations can be performed automatically by the unmannedvehicle (i.e., without a user having to manually remote operate theunmanned vehicle). This can be useful to extend the range of theunmanned vehicle (which normally would be limited based on the user'ssight), thereby saving time and travel by the marine vessel. Automationalso allows the user to perform other tasks.

Further, integration with the marine electronic device 105 allows fordisplay of information regarding the unmanned vehicle on one or moredisplays controlled by the marine electronic device. Thus, informationcan be displayed where most useful for a user, such as on a display thatalso offers other pertinent information about operating the marinevessel or using various features thereof. For example, as will bedescribed in greater detail herein, the position of the unmanned vehiclemay be displayed with respect to the marine vessel on a chart on thescreen of the marine electronic device 105. In another example, sonardata from the unmanned vehicle may be displayed on the same screen as achart showing the position of the unmanned vehicle relative to themarine vessel. In a further example, the user is able to toggle betweena video streaming in from the unmanned vehicle and motor/operationdetails for the marine vessel itself (such as may be used for drivingthe marine vessel safely).

In some embodiments, such operation (e.g., manual remote control orsetting up future autonomous control) of the unmanned vehicle 120 cantake place by the user directly interacting with the marine electronicdevice 105 or indirectly interacting with the marine electronic device105 through a remote device (such as a mobile device/smart phone 109).

Navigation

In some embodiments, the unmanned vehicle 120 may include a locationsensor 128 and/or navigation system. For example, the unmanned vehiclemay be configured to utilize the location sensor 128 to determinelocation data indicative of a current location of the unmanned vehicle120. Additionally, in some embodiments, the unmanned vehicle may beconfigured to determine a location corresponding to a desired locationsuch that it knows how to travel to the desired location (e.g., theunmanned vehicle may determine a relative position of a desired locationversus its current location to determine the direction (includingaltitude, height, depth, etc.) to travel to reach the desired location).

In some embodiments, the unmanned vehicle 120 may be configured tocommunicate with and/or be controlled/operated by a marine electronicdevice 105. In such a configuration, the marine electronic device 105may be configured to utilize one or more features (such as itsnavigation system and stored data therein) to control and/or operate theunmanned vehicle.

In some embodiments, the unmanned vehicle 120 may be configured totransmit the location data indicative of its current location to amarine electronic device 105. With the location data, the marineelectronic device 105 may be configured to determine the currentlocation of the unmanned vehicle.

Additionally, in some embodiments, the marine electronic device 105 maybe configured to determine other information regarding the location dataprovided by the unmanned vehicle, such as the distance to the locationdata, direction to the location data, weather or other informationregarding the current location of the unmanned vehicle, etc. Notably,depending on the desired information/data, constant updating may need tooccur as the marine vessel travels (since the base of operation of theunmanned vehicle moves on the body of water). Further, in someembodiments, marine environment information may be taken into account.For example, when measuring a distance to the unmanned vehicle, thedistance may be measured according to travel time of the marine vesselto that location (which may include traveling around rocks or otherhazards).

Further, along these lines, and in conjunction with additionalfunctionality of the marine electronic device 105 (such as described ingreater detail herein), some embodiments contemplate providingadditional information to the user. For example, the marine electronicdevice 105 may interact with components of the marine vessel todetermine data pertinent for a user to decide whether to travel to thelocation of the unmanned vehicle 120 and/or how to operate/control theunmanned vehicle. Such pertinent data may include, for example, fuellevel, expected fuel used to travel to the location, battery life and/orfuel of the unmanned vehicle (which may indicate the need to travel tothe unmanned vehicle to pick it up), time associated with traveling tothe location of the unmanned vehicle, time associated with the unmannedvehicle returning to the marine vessel, intercept time with both themarine vessel and the unmanned vehicle traveling, among others. Suchpertinent data may be displayed to the user to enable the user to decidehow to operate the marine vessel and/or the unmanned vehicle.

For example, in some embodiments, the marine electronic device 105 maytrack the unmanned vehicle in relation to the marine vessel (e.g., itsbase) and provide an indication to a user if the unmanned vehicle has(or will soon) exceed its range of operation (which can be determinedbased on communication range, fuel gauge, battery life, wind conditions,etc.). Further, information regarding corrective measures (e.g., whereto send the unmanned vehicle and/or marine vessel) may be provided tothe user and/or propulsion systems of the marine vessel and/or unmannedvehicle. Such corrective measures may include sending the unmannedvehicle to another location such as for a safe landing (e.g., on a dock,in shallow water, near another marine vessel, etc.).

In some embodiments, the marine electronic device 105 may be configuredto cause display of navigation or other related data on a screen (e.g.,display 114). In this regard, in some embodiments, the marine electronicdevice 105 may be configured to display the current location of themarine vessel with respect to one or more unmanned vehicles (e.g., fromlocation data gathered from the unmanned vehicle(s)).

For example, FIG. 3A shows an example marine electronic device 40 thatincludes a user interface 42 with a screen 45 and one or morebuttons/knobs (e.g., knob 44, buttons 46 a, 46 b, and 46 c). Thebuttons/knobs may, in some configurations, be utilized by a user tocontrol operation of the marine electronic device 105 (and/or unmannedvehicle(s) by extension including direct control of the propulsionsystem of the unmanned vehicle). Additionally or alternatively, the userinterface 42 may be configured to receive user input directly to thescreen 45 such that it is touch screen.

FIG. 3A depicts a chart 200 that shows a body of water 205 and relativeland 202 around the body of water (including the island in the middle ofthe body of water). The chart 200 also shows a representation of themarine vessel 210 at a current location on the body of water 205.Additionally, such as based on received location data, the chart 200shows representations of one or more unmanned vehicles 220. In someembodiments, such as in the situation where there are more than oneunmanned vehicle, the chart 200 may be configured to provide adistinction between the unmanned vehicles such as with numbers 1, 2,etc. (though other distinguishing display characteristics arecontemplated—including, for example, colors, patterns, symbols, names,etc.). In this regard, a user can easily determine the relative locationof the unmanned vehicle with respect to the marine vessel and/or body ofwater (even in the circumstance when the unmanned vehicle is otherwisenot visible to the user directly).

In some embodiments, the marine electronic device 105 may be configuredto include or act as a navigation system. In this regard, the marineelectronic device 105 may utilize different components (e.g., one ormore processors, one or more various sensors, memory, user interface,etc.) to store, create, and/or determine various navigation relatedfeatures, such as routes, waypoints, survey/scan areas, among others. Insome embodiments, navigation data that is stored and/or utilized by themarine electronic device 105 for use by the marine vessel may also beutilized for operation and control of the unmanned vehicle. In thisregard, the unmanned vehicle 120 can be controlled autonomously and/orremotely operated using the pre-stored navigation data—increasingfunctionality and ease of use by a user.

In such a regard, in some embodiments, the marine electronic device 105may be configured to determine one or more desired location(s) for theunmanned vehicle 120 based on a planned route or waypoint stored in thememory of the marine electronic device 105. In some embodiments, themarine electronic device 105 may be configured to use the determinedcurrent location of the unmanned vehicle 120 in that determination ofthe desired location(s)—such as to check the current location of theunmanned vehicle 120 along a planned route and/or track the unmannedvehicle 120 with respect to the navigation related data.

In some embodiments, the marine electronic device 105 may be configuredto determine one or more routes for travel by the unmanned vehicle. Suchroutes (and/or a single desired location) may be determined based onpre-stored routes or waypoints, current gathered information (e.g.,weather data, sonar data, current operations of the unmanned vehicle(e.g., tracking an object, returning to the marine vessel, interceptinganother marine vessel or unmanned vehicle), among others). For example,the marine electronic device 105 may utilize information gathered froman automatic identification system (AIS) to form a route for theunmanned vehicle such as to intercept another marine vessel and/orunmanned vehicle whose location was determined from the AIS data. Inanother example, the marine electronic device 105 may utilize thecurrent location (and/or speed, heading, etc.) of the marine vessel(which has the marine electronic device 105 attached) to form a routefor the unmanned vehicle 120 such that it may return to the marinevessel. In some embodiments, such as when the marine vessel isstationary, the marine electronic device 105 may simply provide thecurrent location of the marine vessel to the unmanned vehicle to causeit to return to the marine vessel. In some embodiments, the marineelectronic device 105 may be configured to cause the unmanned vehicle105 to return to the marine vessel based on user input indicating such adesire—such as by a user selecting a “Get Drone Back” option (or thelike).

In some embodiments, the marine electronic device 105 may be configuredto determine one or more routes for the unmanned vehicle 120 based onuser input provided to the marine electronic device 105 defining the oneor more routes. In this regard, a user can interact with the marineelectronic device 105 to create a route for the unmanned vehicle 120.

In some embodiments, the marine electronic device 105 may be configuredto generate one or more waypoints based on location data and/or otherdata provided by the unmanned vehicle 120. In some embodiments, thewaypoint may be inputted or caused to be generated through user input tothe marine electronic device 105.

In some embodiments, the marine electronic device 105 may be configuredto form and/or transmit one or more control signals with instructionsfor the unmanned vehicle. In some embodiments, the control signal mayinclude navigation related instructions such as providing instructionsto the unmanned vehicle 120 to travel to a determined desired location.Depending on the configuration and/or the desired outcome, the controlsignal may include any type of instructions for traveling to a desiredlocation, such as any type of desired location data (e.g., coordinates,latitude and longitude, direction and speed data for the unmannedvehicle, etc.). Further, in some embodiments, multiple locations may besent as a single set of instructions to the unmanned vehicle 120 tocause the unmanned vehicle 120 to travel along a route or as part of asurvey/scan (described in greater detail below). In such an embodiment,the unmanned vehicle 120 may include a navigation system onboard suchthat it can travel without further instruction from the marineelectronic device 105. Alternatively, in some embodiments, a series ofcontrol signals may be transmitted to the unmanned vehicle 120 to causethe unmanned vehicle 120 to travel along a route. In some such cases,each consecutive control signal may be sent as the marine electronicdevice 105 determines that the unmanned vehicle 120 has reached acertain point (e.g., a waypoint) along the route.

In some embodiments, the marine electronic device 105 may be configuredto transmit the one or more control signals to the one or more unmannedvehicle 120. Such control signal(s) may be transmitted automatically orin response to user input. For example, a user may interact with awaypoint of a chart. A drop down menu could display that includes aselection for sending a drone to the waypoint. Upon selection by theuser, the unmanned vehicle 120 may travel to the location of thewaypoint based on instructions sent by the marine electronic device 105.Similar methodology could be applied for any type of navigation relatedinstruction (e.g., routes, survey patterns, etc.).

In some embodiments, the marine electronic device 105 may providenavigation instructions to the user for remote control of the unmannedvehicle 105. Such instructions may be displayed on the display of themarine electronic device 105 and/or on any other display (such as thedisplay of a mobile device) for use by the operator of the unmannedvehicle 120 to travel according to the instructions.

In some embodiments, the unmanned vehicle 120 may be configured toreceive the one or more control signals with instructions for travellingto one or more desired locations. In response, the unmanned vehicle 120may cause the propulsion system to propel the unmanned vehicle accordingto the instructions, such as toward the desired location.

In some embodiments, the control signal may include additionalnavigation-related instructions regarding how to travel to the desiredlocation (e.g., speed, altitude, operate component(s) while traveling,operate component(s) upon arrival, etc.). In response to receiving thecontrol signal with the additional instructions, the unmanned vehicle120 may cause it(s) various systems (e.g., the propulsion system,operational components, etc.) to operate accordingly to cause theunmanned vehicle 120 to travel to the desired location (and/or operate)according to those instructions. In some embodiments, the marineelectronic device 105 may be configured to track the unmanned vehicle105 such as it travels according to the instructions from the one ormore control signals.

In some embodiments, the marine electronic device 105 may be configuredto display the route and/or waypoints with respect to the unmannedvehicle 120 (and/or the marine vessel) for use by the operator. Suchdisplay may occur on a screen of the marine electronic device 105 or anyother screen (e.g., a screen of a mobile device of the user). FIG. 3Bshows one such example. In particular, FIG. 3B depicts a chart 200 thatshows a body of water 205 and relative land 202 around the body ofwater. The chart 200 also shows a representation of the marine vessel210 at a current location on the body of water 205. Additionally, suchas based on received location data, the chart 200 shows a representationof an unmanned vehicle 220. Further, the chart 200 shows a portion of aroute 230 that the unmanned vehicle 220 has traveled (shown in solidline) and a planned/proposed portion of the route 232 that the unmannedvehicle 220 is scheduled to travel (shown in dotted line). In thisregard, a user of the marine electronic device 105 may easily track thecurrent and planned travel of the unmanned vehicle 120. Though thedepicted embodiment uses separate patterns for portions of the route,other display types are contemplated for showing the variousnavigation-related features.

In some embodiments, the marine electronic device 105 may be configuredto display other data regarding the unmanned vehicle 120 (or its travel)to the user. For example, the marine electronic device 105 may determine(e.g., through its own sensors/systems or from the unmanned vehicle 120)information (e.g., navigation related, operation related, etc.) aboutthe unmanned vehicle. For example, the marine electronic device 105 maydetermine and display information such the speed of the unmannedvehicle, altitude, current status of any operational components of theunmanned vehicle, current direction of travel of the unmanned vehicle,fuel gauge or other propulsion system information of the unmannedvehicle, among others.

In some embodiments, the marine electronic device 105 may be configuredto determine one or more survey or scan patterns for one or moreunmanned vehicles 120. Such determination may be based on navigationdata stored on the memory of the marine electronic device. In someembodiments, the user may interact with the marine electronic device 105to create the survey/scan patterns. For example, the user may draw thesurvey/scan pattern on the user interface for each unmanned vehicle 120(such as with respect to the chart). In some embodiments, thesurvey/scan pattern may simply be selected based on pre-set patterns. Asdescribed in greater detail herein, the unmanned vehicle 120 may operateone or more of its operational (or other) components while traveling thesurvey/scan patterns.

In some embodiments, the marine electronic device 105 may be configuredto display the various survey/scan patterns along with the currentlocation(s) of the one or more unmanned vehicles 120. FIG. 3C shows anexample chart 200 with representations for the marine vessel 210 and twounmanned vehicles 220. Additionally, the chart 200 shows survey patterns238 for each unmanned vehicle 220. As shown in the depicted embodiment,the survey patterns 238 can vary (for example, the survey pattern on theright has been determined such the unmanned vehicle travels near anisland of the body of water). As will be described in more detailherein, the unmanned vehicle(s) 120 may be configured to gatheroperational data and provide the operational data to the marineelectronic device 105 as they travel along the survey pattern.

In some embodiments, the marine electronic device 105 may be configuredto cause the marine vessel to track, follow, or go to the unmannedvehicle 120. In such an embodiment, the marine electronic device 105 maydetermine the current location of the unmanned vehicle 120 and provideinstructions to one or more autopilots (or an operator via a display) toenable/cause the marine vessel to travel toward/after the unmannedvehicle 120. As will be described in greater detail herein, such anembodiment may be even more robust by utilizing the variousfunctionality of the unmanned vehicle. For example the unmanned vehicle120 may be tracking an object, detecting fish, checking depths ahead ofthe marine vessel, checking wind conditions, scouting, etc.

Landing Operations

In some embodiments, the unmanned vehicle 120 may be configured to landon a marine vessel, such as the marine vessel associated with the marineelectronic device 105. However, landing on a marine vessel in a marineenvironment (or elsewhere, such as on the water surface or other objectin the marine environment) can be difficult and many challenges arepresented. For example, the marine vessel may be moving itself (whetherby its own propulsion system or by wave activity). Additionally, marinevessels tend to have limited open space on their decks or with respectto their main cabins—thereby providing limited landing space. Moreover,there may be obstacles (such as antennas, fly bridges, etc.) that mayblock certain paths or altitudes during the landing operation. Furtherstill, remote control operation during landing may be difficult, asvisibility may be limited by the operator due to any number of factors(limited space on the marine vessel, limited maneuverability of theoperator on the marine vessel, location of the marine electronic devicewith respect to the landing pad, weather, etc.). Likewise, if theunmanned vehicle is landing on the water surface, the surface itself maybe choppy (such as from increased wave activity), thereby making alanding on the water surface difficult. This, like on the marine vessel,may be complicated by decreased visibility of an operator due todistance or other factors (such as weather). As such, some embodimentsof the present invention contemplate configurations of the unmannedvehicle 120 and/or marine electronic device 105 that enable easy landingin a marine environment, including on a marine vessel.

In some embodiments, the marine electronic device 105 is configured todetermine that the unmanned vehicle 120 should perform a landingoperation. In some embodiments, the marine electronic device 105 isconfigured to receive an indication by a user of a desire to land theunmanned vehicle 120. Such an indication, in some embodiments, may beprovided to the user interface of the marine electronic device 105 orfrom a remote device interacting with the marine electronic device 105(e.g., a mobile device). Alternatively, the marine electronic device 105may determine based on stored protocol or a current situation that theunmanned vehicle 120 should perform a landing operation. For example,the marine electronic device 105 may determine that the unmanned vehicle120 has completed scheduled tasks and/or travel and has reached the nextoperation of landing. In another example, the marine electronic device105 may determine that the unmanned vehicle 120 has a low fuel orbattery level (e.g., below a predetermined threshold) and mayautomatically initiate its return to the marine vessel and/or a landingoperation. Additionally or alternatively, in some embodiments theunmanned vehicle 120 may itself determine that is should preform alanding operation (such as through direct remote operation by a user(without the marine electronic device 105) and/or in response to certainsituations (such as those described above or others)).

In some embodiments, the marine electronic device 105 may be configuredto transmit a control signal to the unmanned vehicle 120 withinstructions to perform a landing operation. Depending on the situationand the configuration of the unmanned vehicle and/or marine environment,the landing operation may be performed anywhere. For example, theunmanned vehicle 120 may be instructed or may determine to land on thewater surface. In another example, the unmanned vehicle 120 may beinstructed or may determine to land on a dock or other object. Veryoften, however, the unmanned vehicle 120 may be instructed or maydetermine to land on a marine vessel (e.g., the marine vessel associatedwith the marine electronic device 105). However, as noted above, therecan be many difficulties with landing in a marine environment.

In order to account for some of the difficulties in landing on a marinevessel, in some embodiments, the marine electronic device 105 may beconfigured to determine certain information with respect to the marinevessel or its current location. For example, the marine electronicdevice 105 may be configured to determine a location where the landingoperation is to take place (which may be the current location of themarine vessel or a future location of the marine vessel (e.g., themarine vessel and unmanned vehicle may intercept)). Additionally, themarine electronic device 105 may be configured to determine otherpertinent landing related information regarding the marine vessel, suchas the speed of travel of the marine vessel, the direction of travel ofthe marine vessel, the orientation of the marine vessel, position,altitude, and/or orientation of the landing pad on the marine vessel,applicable landing zone (described in greater detail below), amongothers. Additionally or alternatively, the marine electronic device 105may be configured to determine further pertinent landing relatedinformation corresponding to the planned location of the landingoperation, such as the weather, the sea level, wind conditions, nearbyhazards or objects (which may block an approach vector), among others.

In some embodiments, the marine electronic device 105 may be configuredto determine instructions to provide to the unmanned vehicle 120 inorder for the unmanned vehicle 120 to perform the landing operation. Forexample, such instructions may include instructions on the appropriatelanding approach for the unmanned vehicle 120 to take. This couldinclude any number of different instructions, including, for example,where to travel, how fast to travel, the altitude to travel at, when tochange altitude, when to slow down, the orientation that the unmannedvehicle 120 needs to take during the approach, among many others.

In some embodiments, such instructions may take into account one or moreof the location of the landing operation and additional determinedpertinent landing-related information (e.g., for the marine vesseland/or the location). In some embodiments, the marine electronic device105 is configured to transmit a control signal with the instructions tothe unmanned vehicle 120 to cause it to perform the landing operationaccordingly. In some embodiments, the marine electronic device 105 maytransmit a control signal that includes at least some of the gatheredlanding-related information, and the determination of the operationsneeded to perform the landing operation by the unmanned vehicle 120 canbe performed on the unmanned vehicle 120.

In response to receiving the control signal with the instructions forperforming the landing operation (which may include landing approachinstructions), the unmanned vehicle 120 may be configured to perform thelanding operation. In some embodiments, the landing operation may beperformed using various systems of the unmanned vehicle 120, such as thepropulsion system, other system(s) or sensor(s), etc. Depending on theinstructions, the landing operation may take into account additionallanding-related information (as described above). In such a regard, theunmanned vehicle 120 may cause its systems to perform the landingoperation by propelling the unmanned vehicle 120 based on suchlanding-related information (e.g., the location of the marine vessel,the speed of travel of the marine vessel, the direction of travel of themarine vessel, etc.).

In some embodiments, the unmanned vehicle 120 may be configured to landon the marine vessel, such as on a deck, on a fly bridge, in a dockingstation, on a landing pad, etc. Depending on the configuration, thelanding pad/area for the unmanned vehicle 120 may be any position and/ororientation on the marine vessel. In this regard, in some embodiments,the unmanned vehicle 120 may be configured to land and/or dock in astorage or partial storage configuration such that the landing pad/areais designed to receive the unmanned vehicle 120 in a certain orientation(or various certain orientations). Further, depending on the type andsize of the marine vessel, the landing pad/area may be at a certainaltitude that can be accounted for during landing. For example, theheight of the marine vessel may be large and the landing pad/area may bepositioned at a certain altitude or height (such as on a fly bridge orcaptain's deck), which may be accounted for when instructions areprovided to the unmanned vehicle 20 for landing.

FIG. 4 depicts a marine vessel 10 and unmanned vehicle 20 in a marineenvironment 201. In the depicted example, the marine vessel 10 has alanding pad 227 near the front of the marine vessel 10. However, thelanding pad 227 is positioned on the marine vessel 10 such that it isoriented slightly differently than directly forward (from the marinevessel's perspective). As shown in FIG. 4, the marine vessel 10 istraveling at a speed and in a direction as indicated by arrow 207 (suchas using a motor 17).

In order to perform a landing operation, the unmanned vehicle 20 musttake into account a number of things. For example, the unmanned vehicle20 must account for (and perhaps match) the direction and speed of themarine vessel 10 so that a successful landing can occur. For example,the marine vessel 10 may be moving, and so in order to set down smoothlyor dock smoothly, the unmanned vessel 10 may approach the marine vesselsuch that it is travelling at or close to the speed of the marine vesseland in the direction of travel of the marine vessel. This enables minorcorrections or variations to be performed by the unmanned vehicle 120 toaccomplish a smooth landing.

Further, however, as indicated above, the landing pad/area of the marinevessel may require the unmanned vehicle 120 to land in a certainorientation. In this regard, the instructions may need to account forchanging the orientation of the unmanned vehicle with respect to themarine vessel so as to align with the orientation of the landingpad/area.

Notably, however, it is important to consider the difference betweenusing a marine vessel centric system (e.g., consideringmovement/speed/direction/orientation from the perspective of the marinevessel) and a geo centric system (e.g., consideringmovement/speed/direction/orientation from the perspective of the earth).Indeed, this difference must be taken into account when providinginstructions to the unmanned vehicle 120 for landing. For example, ifthe both the unmanned vehicle 120 and the marine vessel with the marineelectronic device 105 are traveling along, there may only need to be aslight change in location or altitude between the two in order toperform the landing operation (i.e., in the case where the unmannedvehicle is close to landing). However, when providing instructions tothe unmanned vehicle, some embodiments account for thespeed/direction/orientation it is currently traveling. To explain, theinstructions may need to figure out how to maneuver the unmanned vehicle120 to account for the slight change, but still account for the movingmarine vessel (as the desired landing spot is still moving from a geocentric standpoint). In this regard, there is a conversion from a marinevessel centric system to a geo centric system.

In some embodiments, in order to aid in performance of the landingoperation, the unmanned vehicle 120 and/or the landing pad/marine vesselmay have one or more landing sensors. In some embodiments, the landingsensors may be configured to monitor a proximate position of portions ofthe unmanned vehicle 120 and provide feedback to aid in a properlanding. For example, with reference to FIG. 4, the unmanned vehicle 120may have a housing with four landing sensors 29 a-d. Each landing sensor29 a-d may be associated with a corner of the unmanned vehicle 20 suchthat an outline of the general footprint of the unmanned vehicle 20 ismonitored. Likewise, the landing pad 227 may have corresponding landingsensors 229 a-d that correspond to the landing sensors 29 a-d on theunmanned vehicle 20. During the landing operation, the landing sensorscan be utilized to achieve a proper landing. In this regard, in someembodiments, general instructions can be provided to the unmannedvehicle 120 to operate it to get “close” to the landing pad, and thenthe feedback from the landing sensors can be used to finalize thelanding.

Though the above described example includes matching landing sensorsbetween an unmanned vehicle and a landing pad for aiding in landing,other types of sensors or systems are contemplated by embodiments of thepresent invention. Further, the sensors or systems can be positioned onthe unmanned vehicle, on the marine vessel, proximate the landing pad,or combinations thereof. An additional example sensor or system to aidin landing includes one or more optical sensors that can be used withobject recognition to aid in instructing the unmanned vehicle 120 forachieving a proper landing. Along similar lines, in another example, oneor more visual markers can be positioned on the marine vessel such thatthe unmanned vehicle (through object recognition) can discern and usethe visual markers to aid in landing. Another example sensor or systemto aid in landing includes utilizing light beams with distinguishablecolors (e.g., red and green) to provide visually recognizableinstructions or information (e.g., the unmanned vehicle is approachingfrom the right).

Depending on the position of the landing area on the marine vesseland/or configuration of the marine vessel, there may be obstacles orother considerations that make it undesirable/impossible for theunmanned vehicle 120 to approach the marine vessel from certain angles,altitudes, directions, etc. For example, there may be a mast that wouldprevent approach by an unmanned vehicle 120 from the rear of the marinevessel. Similarly, the user may be fishing off to one side of the marinevessel and may want the unmanned vehicle to avoid this area duringlanding or otherwise. As such, some embodiments of the present inventioncontemplate a landing zone that defines a safe approach/landing area orvolume. Such landing zone may include any safe zone definitionsincluding, for example, safe altitudes, minimum altitude, angles ofapproach, combinations thereof, etc. In this regard, the safe landingzone may take into account a three-dimensional space—as an unmannedaerial vehicle may approach from various altitudes.

In some embodiments, the marine electronic device 105 may be configuredto take into account the safe landing zone when providing instructionsto control or operate the unmanned vehicle 120. Additionally oralternatively, the unmanned vehicle 120 may determine where or how totravel or operate by taking the safe landing zone into account (whichmay be provided to the unmanned vehicle 120 from the marine electronicdevice 105).

FIG. 5 illustrates an example safe landing zone 257 around a landing pad227 of a marine vessel 10 in a marine environment 201. In such anembodiment, the unmanned vehicle 20 may only be configured to approachthe landing pad from within the safe landing zone 257.

Though the above description focuses on a safe landing zone, embodimentsof the present invention also contemplate safe zones where the unmannedvehicle 120 will or will not (depending on the configuration) travel,such as during any operation (e.g., not just landing) or other specificoperations. For example, a user may define a “dangerous” zone instead ofa “safe” zone.

In some embodiments, the marine electronic device 105 may be configuredto determine the safe landing zone. In this regard, the safe landingzone may be preset or customizable. For example, a user may beconfigured to define the safe landing zone. FIG. 6 shows an examplemarine electronic device 40 with a screen 45 that shows an interfacethat enables a user to draw the desired safe landing zone. For example,a user 70 may interact with the touch screen with their finger 71 todraw a line 272 from the landing pad 267 of the marine vessel 261. Theline 272 may be formed into the safe landing zone 277. In this regard,in some embodiments, the marine electronic device 105 may be configuredto complete the safe landing zone that is being drawn or created by theuser.

Although the depicted embodiment shows a user drawing a safe landingzone, other user input forms are contemplated. Further, as indicatedabove, a safe zone (such as any operation or all operations) can bedrawn or indicated for use by the marine electronic device 105 andunmanned vehicle 120.

In some embodiments, the unmanned vehicle 120 may be configured to landon the surface of the body of water. As noted herein, in someembodiments, the unmanned vehicle 120 may be configured to float on thesurface of the body of water. In some embodiments, the marine electronicdevice 105 may be configured to transmit a control signal withinstructions for the unmanned vehicle 120 to land on the surface of thebody of water at a location. Such instructions may include (or be basedon) pertinent information regarding the location and/or surface of thebody of water. For example, the marine electronic device 105 maydetermine the current sea level or other characteristics about the sea(wave height, etc.) that can be provided to the unmanned vehicle 120and/or accounted for when forming the instructions for landing on thesurface of the body of water. Accordingly, the unmanned vehicle 120 maybe configured to receive the instructions and cause the unmanned vehicle120 to land on the surface of the body of water (such as using thepropulsion system of the unmanned vehicle).

In some embodiments, the unmanned vehicle 120 may be configured to useobject recognition to avoid obstacles, such as buoys or debris in thewater, when attempting to land on the surface of the body of water.Additionally or alternatively, the unmanned vehicle 120 may beconfigured to provide video data to the marine electronic device 105 toperform the object recognition and instruct the unmanned vehicleaccordingly.

Though the above described embodiments contemplate landing on a marinevessel or the surface of the water, some embodiments of the presentinvention contemplate landing on other objects, including for example,other objects in the water, islands, sandbars, etc. Landing accordinglymay, in some embodiments, occur in response to the unmanned vehicle 120receiving instructions from the marine electronic device 105.

Operational Control

As noted above, in some embodiments, the unmanned vehicle 120 may beconfigured to communicate with and/or be controlled/operated by themarine electronic device 105. In such embodiments, the unmanned vehicle120 may be configured to receive instructions from the marine electronicdevice 105 to gather operational data and/or perform various operationswith systems, sensors, or other components of the unmanned vehicle 120.In response, the unmanned vehicle 120 may act accordingly and provideoperational data and/or data indicating completion of various operationsto the marine electronic device 105. As used herein, operational datamay encompass any type of data that the unmanned vehicle can gather(e.g., sonar data, camera data, location data, etc.). Additionally, someembodiments of the present invention utilize further categorization ofdata types, such as sensor data, which may refer to any type of datacapable of being gathered by a sensor of the unmanned vehicle, which mayinclude sonar, radar, location sensors, among others. Also, as usedherein, operations that can be performed by the unmanned vehicle 120 mayrefer to any type of operation described or contemplated herein (e.g.,tasks, travel, transmission, receipt, gather data, etc.).

Video/Camera Operation

In some embodiments, the unmanned vehicle 120 may include a camera 133configured to gather camera data, including, for example, video and/orimage data.

In some embodiments, the unmanned vehicle 120 may be configured tocommunicate with and/or be controlled/operated by the marine electronicdevice 105. In such embodiments, the unmanned vehicle 120 may beconfigured to operate the camera 133 according to instructions providedby the marine electronic device 105 (such as through a control signal).Likewise, in some embodiments, the unmanned vehicle 120 may beconfigured to provide the gathered camera data to the marine electronicdevice 105. This may include streaming video and/or still images. Insome embodiments, the gathered camera data could be provided through adirect transmission to the marine electronic device 105, through anexternal network, or via upload after return of the unmanned vehicle120.

In some embodiments, the marine electronic device 105 may be configuredto display the camera data (e.g., the streaming video) to a user, suchas on a screen. Notably, display on the screen of the marine electronicdevice 105 may be desirable to provide a large screen and/or convenientviewing area (e.g., the command center of the boat, where the user isfishing, etc.). Further, use of such screens may be desirable as theyare often high-bright screens that are designed for easy viewing insunlight (since the screens are often exposed on the marine vessel).This allows for easy viewing of objects, such as fish, sand bars, rocks,bridges, etc.

Indeed, in some embodiments, the unmanned vehicle 120 (such as byutilizing the camera and displaying the resulting image on the marineelectronic device 105) may be used to scout the marine environment, suchas for depth, possible fishing locations, among many other reasons. Forexample, in some embodiments, the unmanned vehicle 120 may be instructedto act as a virtual tuna tower. In such a regard, the user may provideuser input indicating that the unmanned vehicle 120 should perform a“virtual tuna tower” operation. In response, the marine electronicdevice 105 may transmit a control signal with instructions for theunmanned vehicle 120 to fly to a specified height and provide streamingvideo back to the marine electronic device 105 to aid in a user spottingfish (e.g., tuna). In some embodiments, the unmanned vehicle 120 may befurther configured to travel with the marine vessel (such as in responseto instructions to do so) such that it stays in the same place relativeto the marine vessel—acting as a “virtual tuna tower.” As furtherdetailed herein, object recognition technology (such as at the marineelectronic device 105 or the unmanned vehicle 120) could be used tofurther aid in fish spotting.

Additionally, in some embodiments, the unmanned vehicle 120 may beconfigured to determine location data associated with the gatheredcamera data. Further, in some embodiments, the unmanned vehicle 120 maybe configured to transmit the location data to the marine electronicdevice 105, such as for determination of the location of the unmannedvehicle at the time of gathering the camera data (though the locationmay also or in the alternative be determined at the unmanned vehicle).

In some embodiments, the location of the unmanned vehicle 120 may beassociated with the gathered camera data for correlation and/or storage.Such association may occur at the unmanned vehicle 120 and/or at themarine electronic device 105. In this regard, in some embodiments, themarine electronic device 105 may cause display of the streaming video inassociation with the location of the unmanned vehicle to give context tothe streaming video to the user.

FIG. 7A shows an example marine electronic device 40 that depicts asplit-screen view 300 of streaming video 301 on the left portion of thescreen and a chart 303 on the right portion of the screen. The streamingvideo 301 is taken from camera data from the unmanned vehicle 120 andshows a bird's eye view (such as from a flying unmanned vehicle 120) ofwater 311 with a fish 313 swimming near the surface of the water. Thechart 303 shows the location of unmanned vehicle 320 (indicated as a“D¹”, such as for Drone 1) on the body of water 305. The chart 303 alsoshows the location of the marine vessel 310 associated with the marineelectronic device 105, thereby giving the viewer an indication of therelative position of the unmanned vehicle 120.

Additionally, in some embodiments, such as the depicted embodiment, themarine electronic device 105 may display an indication of the cameraview 351 that provides the user with an idea of the orientation and/orcoverage area of the camera and the captured streaming video. In suchembodiments, the unmanned vehicle 120 may be configured to determineorientation data corresponding to the orientation of the unmannedvehicle 120 and/or the camera 133 (as the camera 133 may be configuredto be directed with respect to the unmanned vehicle 120). Suchorientation data may be provided to the marine electronic device 105 foruse (such as for display, association with the camera data, etc.). Inthis regard, a user can easily determine the relative location andorientation of the streaming video with respect to the marine vesseland/or body of water (even in the circumstance when the unmanned vehicleis otherwise not visible to the user directly).

In some embodiments, the marine electronic device 105 may be configuredto overlay the camera data on the chart at the corresponding locationand in the corresponding orientation as the camera data was received.For example, the marine electronic device 105 may receive the cameradata, the orientation data of the camera data, and the location dataassociated with the camera data. Based on this information, the marineelectronic device 105 may be configured to orient the camera data andoverlay it on a chart at the proper associated location for which it wasgathered. In some embodiments, only a portion of the camera data may beoverlaid on the chart. In this regard, depending on the configurationand/or desires of the user, the marine electronic device 105 may beconfigured to crop the camera data prior to overlaying it on the chart.Such embodiments that utilize overlaying camera data on a chart may beuseful for surveying a body of water. Even more, multiple unmannedvehicles could be used to efficiently survey the body of water, andcould be operated in conjunction via the marine electronic device.

FIG. 7B shows an example screen 45 of a marine electronic device 40showing the streaming video 301 on the left portion and a chart 303′ onthe right portion. Prior camera data has been overlaid 353 on the chart303′ at the appropriate location and in the appropriate orientation. Inthe depicted embodiment, the camera data is overlaid as the unmannedvehicle 320 travels along the body of water 305.

In some embodiments, the unmanned vehicle 120 may be configured toperform an “Auto-Selfie” operation where the unmanned vehicle 120follows the marine vessel to provide a selfie of the marine vessel viathe camera data. In this regard, the marine electronic device 105 mayreceive user input indicating a desire to have the unmanned vehicle 120perform the “Auto-Selfie” operation. In response, the marine electronicdevice 105 may determine and provide instructions to the unmannedvehicle 120 (such as a series of control signals) to cause the unmannedvehicle 120 to travel (e.g., fly) to a series of new desired locationssuch that it follows the marine vessel. In this regard, the marineelectronic device 105 takes into account the current location, heading,and speed of the marine vessel when instructing the new desired locationto the unmanned vehicle such that the unmanned vehicle remains in thesame “relative” position to the marine vessel. Accordingly, the unmannedvehicle 120 may operate its propulsion system to travel according to theinstructions.

Additionally, the instructions may include instructions for the unmannedvehicle to orient the camera toward the marine vessel and begingathering camera data. Accordingly, the unmanned vehicle 120 may directand operate its camera 133 according to the instructions. This cameradata may be provided back to the marine electronic device for displayand/or storage—creating a “selfie” of the marine vessel. Suchembodiments may be useful in many different scenarios (such as forlearning to sail, learning to drive a boat, docking, etc.).Additionally, other possible advantages include following a wake boarderor skier to provide action shots. In this regard, various patterns oraltitudes could be used to provide the desired streaming videoexperience. Such instructions could be determined from user input and/orfrom pre-stored patterns. In some embodiments, object recognitiontechnology could be used to enhance the experience by following a skieror other object.

FIG. 8 illustrates an example marine environment where the unmannedvehicle 20 is performing an “Auto-Selfie” operation. In this regard, theunmanned vehicle 20 uses its propulsion system 25 to travel (e.g., alongarrow 362) such that it follows the marine vessel 10 (which istraveling, via a propulsion system 17, along arrow 361). Additionally,the unmanned vehicle 20 has oriented its camera 330 such that the cameraviewing angle 331 covers the marine vessel 10. Such gathered camera datacan then be provided to the marine electronic device 40 for storageand/or display.

In some embodiments, the unmanned vehicle 120 and/or marine electronicdevice 105 may include object recognition technology that can be used inconjunction with the camera data to recognize objects (e.g., fish,dolphins, whales, sharks, other marine vessels, structure, shallowwater, rocks, etc.) and act accordingly. In some embodiments, certainobjects may cause triggers to occur, such as begin capturing camera datawhen a fish or other object is recognized, begin transmission of thecamera data to the marine electronic device for display when a fish orother object is recognized, mark a location on the marine electronicdevice when a fish or other object is recognized, among many others.

Sonar Operation

In some embodiments, the unmanned vehicle 120 may include a sonar system136 configured to gather sonar data. Sonar (SOund Navigation AndRanging) has long been used to detect waterborne or underwater objects.For example, sonar devices may be used to determine depth and bottomtopography, detect fish, locate wreckage, etc. Sonar beams, from atransducer assembly of a sonar system (e.g., sonar system 136), can betransmitted into the underwater environment. The sonar signals reflectoff objects in the underwater environment (e.g., fish, structure, seafloor bottom, etc.) and return to the transducer assembly, whichconverts the sonar returns into sonar data that can be used to producean image of the underwater environment.

In some embodiments, the unmanned vehicle 120 may be configured tocommunicate with and/or be controlled/operated by the marine electronicdevice 105. In such embodiments, the unmanned vehicle 120 may beconfigured to operate the sonar system 136 according to instructionsprovided by the marine electronic device 105 (such as through a controlsignal). Likewise, in some embodiments, the unmanned vehicle 120 may beconfigured to provide the gathered sonar data to the marine electronicdevice 105. In some embodiments, the gathered sonar data could beprovided through a direct transmission to the marine electronic device105, through an external network, or via upload after return of theunmanned vehicle 120. In some embodiments, the unmanned vehicle 120 maybe configured to connect to an external network directly. In suchembodiments, the sonar data may be sent to an external device or serverfor storage and/or use (such as in sonar mapping of the body of water).In some embodiments, the sonar data may be stored in a buffer untilinternet connectivity is reached for transmission (such as to the marineelectronic device 105 and/or external network).

Depending on the configuration of the sonar system 136 of the unmannedvehicle 120 and/or the marine electronic device 105, processing of thesonar returns may occur at either or both of the unmanned vehicle 120 orthe marine electronic device 105. In this regard, in some embodiments,the unmanned vehicle 120 may be configured to merely collect sonarreturns and provide sonar return data to the marine electronic device105 for processing. Additionally or alternatively, the unmanned vehicle120 may be configured to process the sonar return data and generatesonar image data that can be provided to the marine electronic device105. Along these lines, in some embodiments, the unmanned vehicle 120may be configured to detect the presence of a fish or other object fromthe sonar returns and remotely provide a signal to the marine electronicdevice 105. Such a signal may indicate that the captured sonar data mayinclude something of interest. In some embodiments, the unmanned vehicle120 may also be configured to return to the marine electronic device 105and/or marine vessel so that the sonar data containing the possibleobject of interest can be uploaded and/or viewed.

In some embodiments, the marine electronic device 105 may be configuredto display the sonar data to a user, such as on a screen. Notably,display on the screen of the marine electronic device 105 may bedesirable to provide a large screen and/or convenient viewing area(e.g., the command center of the boat, where the user is fishing, etc.).Further, use of such screens may be desirable as they are oftenhigh-bright screens that are designed for easy viewing in sunlight(since the screens are often exposed on the marine vessel).

Additionally, in some embodiments, the unmanned vehicle 120 may beconfigured to determine location data associated with the gathered sonardata. Further, in some embodiments, the unmanned vehicle 120 may beconfigured to transmit the location data to the marine electronic device105, such as for determination of the location of the unmanned vehicleat the time of gathering the sonar data (though the location may also orin the alternative be determined at the unmanned vehicle).

In some embodiments, the location of the unmanned vehicle 120 may beassociated with the gathered sonar data for correlation and/or storage.Such association may occur at the unmanned vehicle 120 and/or at themarine electronic device 105. In this regard, in some embodiments, themarine electronic device 105 may cause display of an image of the sonardata in association with the location of the unmanned vehicle to givecontext to the sonar data to the user.

FIG. 9A shows an example marine electronic device 40 that depicts asplit-screen view 400 of a sonar image 462 on the left portion of thescreen and a chart 402 on the right portion of the screen. The sonarimage 462 is produced from sonar data from the unmanned vehicle 120 andshows sonar imagery taken from a transducer assembly with sidescantransducers. In particular, the sidescan sonar image view shows thesonar view from the left and right of the transducer assembly of thesonar system as time passes (building up from top down like awaterfall). The chart 402 shows the location of unmanned vehicle 420(indicated as a “D¹”, such as for Drone 1) on the body of water 405. Thechart 402 also shows the location of the marine vessel 410 associatedwith the marine electronic device 105, thereby giving the viewer anindication of the relative position of the unmanned vehicle 120.

Additionally, in some embodiments, such as the depicted embodiment, themarine electronic device 105 may display an indication of the area ofcoverage 461 of the sonar system (e.g., the extent to which the sonarbeams from the sonar system provide coverage of the underwaterenvironment). Such an indication would provide the user with an idea ofthe orientation and/or coverage area of the sonar system and thecaptured sonar data. In such embodiments, the unmanned vehicle 120 maybe configured to determine orientation data corresponding to theorientation of the unmanned vehicle 120 and/or the sonar system 136 (asthe sonar system 136 may be configured to be directed with respect tothe unmanned vehicle 120). Such orientation data may be provided to themarine electronic device 105 for use (such as for display, associationwith the sonar data, etc.). In this regard, a user can easily determinethe relative location and orientation of the sonar data with respect tothe marine vessel and/or body of water (even in the circumstance whenthe unmanned vehicle is otherwise not visible to the user directly).

In some embodiments, the unmanned vehicle 120 (such as by utilizing thesonar system 136 and displaying the resulting image on the marineelectronic device 105) may be used to scout the underwater environment,such as for depth determinations, possible fishing locations, among manyother reasons. In some embodiments, the unmanned vehicle 120 may beinstructed to perform search or survey patterns or travel in relation tothe marine vessel—such as to aid a fisherman in finding good fishinglocations, etc. For example, in some embodiments, the unmanned vehicle120 may be configured to scout ahead of the marine vessel to confirm orcheck terrain (e.g., depth) in front of the marine vessel. In thisregard, travel and operation of the sonar system 136 of the unmannedvehicle 120 may be autonomous, such as through control by the marineelectronic device 105 (which can be based on a marine vessel, on shore,at a station, etc.—thereby enabling control of the unmanned vehicle 120in the marine environment from anywhere).

In some embodiments, the marine electronic device 105 may be configuredto scout or survey an area using multiple unmanned vehicles 120. In suchan embodiment, the marine electronic device 105 may track each unmannedvehicle 120 and/or pre-map the survey patterns to control the unmannedvehicles 120 for efficient surveying. In this regard, the marineelectronic device 105 may act as a “hive mind” to control and/or managethe “swarm” of unmanned vehicles 120 to efficiently perform tasks (suchas tasks described herein, including capturing sonar data for a body ofwater, among any other tasks or operations described herein).

In some embodiments, the marine electronic device 105 may be configuredto overlay the sonar data on the chart at the corresponding location andin the corresponding orientation as the sonar data was received. Forexample, the marine electronic device 105 may receive the sonar data,the orientation data of the sonar data, and the location data associatedwith the sonar data. Based on this information, the marine electronicdevice 105 may be configured to orient the sonar data and overlay it ona chart at the proper associated location for which it was gathered.

In some embodiments, only a portion of the sonar data may be overlaid onthe chart. In this regard, depending on the configuration and/or desiresof the user, the marine electronic device 105 may be configured toremove a portion of the sonar data prior to overlaying it on the chart.For example, the marine electronic device 105 may remove the watercolumn of the sonar image data prior to overlaying on the chart suchthat the overlaid image on the chart includes bottom features (whichgives the user an impression of the bottom). Such embodiments thatutilize overlaying sonar data on a chart may be useful for surveying abody of water. Even more, multiple unmanned vehicles could be used toefficiently survey the body of water, and could be operated inconjunction via the marine electronic device 105.

FIG. 9B shows an example screen 45 of a marine electronic device 40showing the sonar image 462 on the left portion and a chart 402 on theright portion. Prior sonar data has been overlaid 463 on the chart 402at the appropriate location and in the appropriate orientation. In thedepicted embodiment, the sonar data is overlaid as the unmanned vehicle420 travels along the body of water 405.

With reference to FIG. 9B, the sonar image 462 is an image from atransducer assembly with sidescan (left and right) transducers. In thisregard, the image includes a centerline 466 that indicates the center ofthe transducer assembly and image data extending out to the left andright of the centerline 466. The sonar data extending outwardly from thecenterline 466 correlates to the distance from which the sonar return isreceived by the transducer assembly (e.g., the distance from thecenterline). As such, there is a water column 469 that includes fish andother objects. Additionally, there is a portion 467 of the screenindicating sonar returns that have returned from the floor of the bodyof water. Notably, the image 462 shows a possible relatively distinctfloor. In some embodiments, the marine electronic device 105 and/orunmanned vehicle 120 may be configured to remove the water column 469 upto that floor. The resulting image could be used for the overlay imageon the chart such that only the topography of the floor is shownoverlaid onto the chart.

As noted herein, the unmanned vehicle 120 may be in many differentconfigurations, such that embodiments of the present inventioncontemplate many different ways to operate a sonar system 136 of anunmanned vehicle 120. In this regard, in some embodiments, the sonarsystem 136 needs to be at least partially submerged in water in order toeffectively collect sonar data. In some embodiments, the unmannedvehicle 120 may be configured for surface travel or submersible travelsuch that the sonar system 136 may be positioned and/or oriented on theunmanned vehicle 120 in a manner in which it is at least partiallysubmerged for operation. In other embodiments, the unmanned vehicle 120may be configured for aerial travel. In this regard, some embodiments ofthe present invention contemplate means to obtain sonar data using anaerial traveling unmanned vehicle. Some examples are provided below.

In some embodiments, the unmanned vehicle 120 may be configured to landand float on or hover just above the surface of the water to enable thesonar system 136 to be at least partially submerged for operation. Sucha maneuver (and operation) could be performed in response toinstructions from the marine electronic device 105. An example of thisis shown in the marine environment 401 of FIG. 10A. In the depictedembodiment, the unmanned vehicle 20 has used its propulsion system toland on the surface 12 of the body of water 11 (such as in response toinstructions from the marine electronic device 40 of the marine vessel10). Notably, the unmanned vehicle 20 includes a sonar system 460 thatis now at least partially submerged such that it can transmit sonarbeams 461 a, 461 b, and 461 c into the underwater environment.

In some embodiments, the unmanned vehicle 120 may be configured to hoverabove the surface of the water (such as at a hover position) and drop ordeploy the sonar system 136 down into the water such that it is at leastpartially submerged for operation. Such a maneuver (and operation) couldbe performed in response to instructions from the marine electronicdevice 105. In some embodiments, the unmanned vehicle 120 may beconfigured pull the sonar system 136 back up for travel and/or travelwith the sonar system deployed (such as during a survey pattern). Anexample of this is shown in the marine environment 401′ of FIG. 10B. Inthe depicted embodiment, the unmanned vehicle 20 has used its propulsionsystem to hover above the surface 12 of the body of water 11 (such as inresponse to instructions from the marine electronic device 40 of themarine vessel 10). Additionally, the unmanned vehicle 20 has lowered(such as via a cable 467) a sonar system 460′ into the water 11. Assuch, the sonar system 460 is now at least partially submerged such thatit can transmit sonar beams 461 a, 461 b, and 461 c into the underwaterenvironment. In some embodiments, the unmanned vehicle 120 may beconfigured to simply drop the sonar system into the water (e.g., withoutit still be attached to a rope or tether).

Radar Operation

In some embodiments, the unmanned vehicle 120 may include a radar system137 configured to gather radar data. In such embodiments, the unmannedvehicle 120 may be utilized to gather radar data for the environmentsurrounding the marine vessel. Indeed, the unmanned vehicle 120 may flyat higher altitudes than typical radar systems for marine vessel, whichmay provide better radar data.

In some embodiments, the unmanned vehicle 120 may be configured tocommunicate with and/or be controlled/operated by the marine electronicdevice 105. In such embodiments, the unmanned vehicle 120 may beconfigured to operate the radar system 137 according to instructionsprovided by the marine electronic device 105 (such as through a controlsignal). Likewise, in some embodiments, the unmanned vehicle 120 may beconfigured to provide the gathered radar data to the marine electronicdevice 105. In some embodiments, the gathered radar data could beprovided through a direct transmission to the marine electronic device105, through an external network, or via upload after return of theunmanned vehicle 120. In some embodiments, the unmanned vehicle 120 maybe configured to connect to an external network directly, such as toprovide radar data to a remote device.

Depending on the configuration of the radar system 137 of the unmannedvehicle 120 and/or the marine electronic device 105, processing of theradar returns may occur at either or both of the unmanned vehicle 120 orthe marine electronic device 105. In this regard, in some embodiments,the unmanned vehicle 120 may be configured to merely collect radarreturns and provide radar return data to the marine electronic device105 for processing. Additionally or alternatively, the unmanned vehicle120 may be configured to process the radar return data and generateradar image data that can be provided to the marine electronic device105.

In some embodiments, the marine electronic device 105 may be configuredto display the radar data to a user, such as on a screen. Notably,display on the screen of the marine electronic device 105 may bedesirable to provide a large screen and/or convenient viewing area(e.g., the command center of the boat, etc.). Further, use of suchscreens may be desirable as they are often high-bright screens that aredesigned for easy viewing in sunlight (since the screens are oftenexposed on the marine vessel).

Additionally, in some embodiments, the unmanned vehicle 120 may beconfigured to determine location data associated with the gathered radardata. Further, in some embodiments, the unmanned vehicle 120 may beconfigured to transmit the location data to the marine electronic device105, such as for determination of the location of the unmanned vehicleat the time of gathering the radar data (though the location may also orin the alternative be determined at the unmanned vehicle).

In some embodiments, the location of the unmanned vehicle 120 may beassociated with the gathered radar data for correlation and/or storage.Such association may occur at the unmanned vehicle 120 and/or at themarine electronic device 105. In this regard, in some embodiments, themarine electronic device 105 may cause display of an image of the radardata in association with the location of the unmanned vehicle to givecontext to the radar data to the user.

FIG. 11 shows an example marine electronic device 40 that depicts asplit-screen view 500 of a radar image 576 on the left portion of thescreen and a chart 502 on the right portion of the screen. The radarimage 576 is produced from radar data from the unmanned vehicle 120. Inparticular, the radar image view shows a center point 572 correspondingto the unmanned vehicle 520. In this regard, a rock or other object 573is shown on the radar image in a corresponding location as shown on thechart 502 with respect to the unmanned vehicle 520. The radar image 576also shows the current radar beam 574 direction. The chart 502 shows thelocation of unmanned vehicle 520 (indicated as a “D¹”, such as for Drone1) on the body of water 505. The chart 502 also shows the location ofthe marine vessel 510 associated with the marine electronic device 105,thereby giving the viewer an indication of the relative position of theunmanned vehicle 120.

Additionally, in some embodiments, such as the depicted embodiment, themarine electronic device 105 may display an indication of the area ofcoverage 571 of the radar system (e.g., the extent to which the radarsystem provides coverage of the surrounding environment). Such anindication would provide the user with an idea of the coverage area ofthe radar system and the captured radar data.

In such embodiments, the unmanned vehicle 120 may be configured todetermine orientation data corresponding to the orientation of theunmanned vehicle 120. Such orientation data may be provided to themarine electronic device 105 for use (such as for display, associationwith the radar data, etc.). For example, the radar display may be “northup”, but the unmanned vehicle 120 may be oriented in a direction otherthan north. Thus, the location of any objects within the radar imagemust be accounted for. Likewise, the orientation of the unmanned vehicle120 may be different than the heading of the marine vessel (on which theuser is viewing the radar image). Thus, the heading of the marinevessel, the orientation of the unmanned vehicle, and/or thegeo-referenced location of the object within the radar data may beprocessed by the marine electronic device 105 to generate an appropriateradar image for display and use. In this regard, a user can easilydetermine the relative location of objects within the radar image withrespect to the marine vessel and/or body of water.

In some embodiments, the marine electronic device 105 may be configuredto process the radar data from the unmanned vehicle 120 for use withradar data taken from the marine vessel (e.g., from a separate radarsystem of the marine vessel). In this regard, the marine electronicdevice 105 may be configured to update radar imagery from the marinevessel's radar system with radar data taken from the unmanned vehicle120. For example, the marine electronic device 105 may updatecharacteristics (e.g., shape, etc.) of objects within the coverage areaof the marine vessel's radar system based on the radar data from theunmanned vehicle 120. In some embodiments, the marine electronic device105 may use the radar data from the unmanned vehicle 120 to formthree-dimensional (3D) radar imagery. In some embodiments, the marineelectronic device 105 may be configured to control and/or operate theunmanned vehicle 120 to help form the desired combined radar data and/orimages (e.g., control the location/pattern of travel to help fill outthe combined radar image and/or 3D image). Along these lines, the marineelectronic device 105 may recognize an object and instruct the unmannedvehicle 120 to scout out or gather more radar information regarding theobject—thereafter updating the radar image accordingly.

In some embodiments, the marine electronic device 105 may be configuredto form a radar map of the surrounding area (e.g., the body of water).In this regard, in some embodiments, the unmanned vehicle 120 (such asby utilizing the radar system 137 and displaying the resulting image onthe marine electronic device 105) may be used to scout the surroundingenvironment, such as for viewing objects, among many other reasons(e.g., fog may prevent/limit visibility). In some embodiments, theunmanned vehicle 120 may be instructed to perform search or surveypatterns or travel in relation to the marine vessel—such as to aid innavigation, etc. For example, in some embodiments, the unmanned vehicle120 may be configured to scout ahead of the marine vessel to confirm orcheck terrain in front of the marine vessel. In this regard, travel andoperation of the radar system 137 of the unmanned vehicle 120 may beautonomous, such as through control by the marine electronic device 105.

In some embodiments, the marine electronic device 105 may be configuredto overlay the radar data (or combined radar data) on the chart at thecorresponding location, such as the radar data was received. Forexample, the marine electronic device 105 may receive the radar data,orientation data of the unmanned vehicle while acquiring the radar data,and the location data associated with the radar data. Based on thisinformation, the marine electronic device 105 may be configured toorient the radar data and overlay it on a chart at the proper associatedlocation for which it was gathered. Such embodiments may be useful informing a radar map of the surrounding environment.

Other Sensor(s)/System(s) Operation

In some embodiments, the unmanned vehicle 120 may include othersystems/sensors (e.g., other systems/sensors 139 of FIG. 19) configuredto gather operational data and/or perform tasks. In some embodiments,the unmanned vehicle 120 may be configured to communicate with and/or becontrolled/operated by the marine electronic device 105. In suchembodiments, the unmanned vehicle 120 may be configured to operate theother systems/sensors 139 according to instructions provided by themarine electronic device 105 (such as through a control signal).Likewise, in some embodiments, the unmanned vehicle 120 may beconfigured to provide the gathered operational data to the marineelectronic device 105. In some embodiments, the gathered operationaldata could be provided through a direct transmission to the marineelectronic device 105, through an external network, or via upload afterreturn of the unmanned vehicle 120. In some embodiments, the unmannedvehicle 120 may be configured to connect to an external networkdirectly, such as to provide operational data to a remote device.

Depending on the configuration of the other systems/sensors 139 of theunmanned vehicle 120 and/or the marine electronic device 105, processingof the operational data may occur at either or both of the unmannedvehicle 120 or the marine electronic device 105. In this regard, in someembodiments, the unmanned vehicle 120 may be configured to merelycollect and provide operational data to the marine electronic device 105for processing. Additionally or alternatively, the unmanned vehicle 120may be configured to process the operational data for use by the marineelectronic device 105 (such as for display). In some embodiments, themarine electronic device 105 may be configured to display theoperational data (or byproducts thereof) to a user, such as on a screen.

Additionally, in some embodiments, the unmanned vehicle 120 may beconfigured to determine location data associated with the gatheredoperational data. Further, in some embodiments, the unmanned vehicle 120may be configured to transmit the location data to the marine electronicdevice 105, such as for determination of the location of the unmannedvehicle at the time of gathering the operational data (though thelocation may also or in the alternative be determined at the unmannedvehicle).

In some embodiments, the location of the unmanned vehicle 120 may beassociated with the gathered operational data for correlation and/orstorage. Such association may occur at the unmanned vehicle 120 and/orat the marine electronic device 105. In this regard, in someembodiments, the marine electronic device 105 may cause display of animage related to the operational data in association with the locationof the unmanned vehicle to give context to the operational data to theuser.

Examples of various other systems/sensors contemplated for use withvarious embodiments of the present invention are provided below. Suchexamples are meant to provide examples and are not meant to be limiting,as embodiments of the present invention contemplate utilization of manydifferent types of systems and/or sensors, such as may be useful withembodiments of the present invention described herein.

In some embodiments, the unmanned vehicle 120 may include a microphonethat is configured to gather audio data (e.g., operational data).Depending on the configuration of the microphone and/or the position ofthe microphone on the unmanned vehicle 120, the unmanned vehicle 120 maygather audio data as it travels (e.g., flies) such as for gatheringnature sounds, bird sounds, wildlife, etc. In some embodiments, theunmanned vehicle 120 may land on the surface of the water, and themicrophone may be configured to gather audio data from the underwaterenvironment.

In some embodiments, the gathered audio data may be provided to themarine electronic device 105 for playing to a user. In some embodiments,the audio data could be associated with a location of the unmannedvehicle 120 when it was acquired. Such information could be utilized ona display (such as described herein with respect to sonar and radardata). Additionally or alternatively, the audio data could be utilizedin other ways (e.g., combining with other operational data, playingnature sounds to the boaters for peaceful sounds, etc.).

In some embodiments, the unmanned vehicle 120 may be configured to trackaudio data such as to locate the source of the audio data. For example,during a search and rescue operation (such as examples describedherein), the unmanned vehicle 120 may utilize the microphone to trackshouting from a person overboard to locate the person. In someembodiments, the unmanned vehicle 120 (such as under control of themarine electronic device 105) may be configured to perform a searchpattern or survey to gather audio data.

In some embodiments, the unmanned vehicle 120 may include a speaker suchas for playing audio data. In some embodiments, the unmanned vehicle 120may include one or more speakers and one or more microphones. In such anembodiment, the unmanned vehicle 120 may act as a 2 way communicationdevice. Such a device may be useful for talking between boats or duringsearch and rescue operations (such as for talking with a personoverboard).

In some embodiments, the unmanned vehicle 120 may include an infraredsensor/camera that is configured to gather infrared data (e.g.,operational data). Depending on the configuration of the infraredsensor/camera and/or the position of the infrared sensor/camera on theunmanned vehicle 120, the unmanned vehicle 120 may gather infrared dataas it travels (e.g., flies). In some embodiments, the unmanned vehicle120 may land on the surface of the water, and the infrared sensor/cameramay be configured to gather infrared data from the underwaterenvironment.

In some embodiments, the gathered infrared data may be provided to themarine electronic device 105 for display to a user. In some embodiments,the infrared data could be associated with a location of the unmannedvehicle 120 when it was acquired. Such information could be utilized ona display (such as described herein with respect to sonar and radardata). Additionally or alternatively, the infrared data could beutilized in other ways (e.g., combining with other operational data,object recognition, etc.).

In some embodiments, the unmanned vehicle 120 may be configured to usethe infrared data to identify people or objects, such as in the water.For example, during a search and rescue operation (such as examplesdescribed herein), the unmanned vehicle 120 may utilize the infraredsensor/camera to locate people during a man overboard situation. In suchembodiments, the unmanned vehicle 120 (such as under control of themarine electronic device 105) may be configured to perform a searchpattern or survey to gather the infrared data. Additionally, in someembodiments, object recognition could be used to autonomously locate aperson in the water. The object recognition could be performed at theunmanned vehicle 120 and/or at the marine electronic device 105. In someembodiments, alerts could be provided to the marine electronic device105 (and/or proper authorities) based on heat signatures and/or possibleidentifications. Additionally or alternatively, locations correspondingto possible heat signature hits could be provided to the marineelectronic device 105, such as for display on a chart. Further, themarine electronic device 105 could automatically determine navigationinstructions for the marine vessel to reach the one or more determinedlocations. An autopilot could even be engaged to automatically travelaccording to the determined navigation instructions.

In some embodiments, the unmanned vehicle 120 may include a temperaturesensor that is configured to gather temperature data (e.g., operationaldata). Such temperature data may be useful for determining optimalfishing locations (e.g., through determining thermoclines, etc.). Insome embodiments, the temperature sensor may include an IR sensor thatis configured to measure sea temperatures.

Depending on the configuration of the temperature sensor and/or theposition of the temperature sensor on the unmanned vehicle 120, theunmanned vehicle 120 may gather atmospheric temperature data (e.g., asit travels) and/or underwater temperature data (e.g., the unmannedvehicle 120 may land on the surface of the water). In some embodiments,the temperature sensor may be formed of a probe (e.g., a telescopicprobe, attached to a line, etc.) that can be deployed within theunderwater environment.

In some embodiments, the gathered temperature data may be provided tothe marine electronic device 105, such as for display to a user. In someembodiments, the temperature data could be associated with a location ofthe unmanned vehicle 120 when it was acquired. Such information could beutilized on a display (such as described herein with respect to sonarand radar data). Additionally or alternatively, the temperature datacould be utilized in other ways (e.g., combining with other operationaldata, object recognition, etc.).

As noted above, in some embodiments, the temperature sensor may beconfigured to measure atmospheric temperature. Likewise, or in addition,the unmanned vehicle 120 may be configured to determine or measureweather data (such as with a temperature sensor, weather antenna, windsensor, camera, etc.). As such, the unmanned vehicle 120 may be utilized(such as by the marine electronic device 105) for determining upcomingweather conditions. In this regard, the unmanned vehicle 120 could flyup to an altitude or out to a specific location to check the weather.

Along similar lines, in some embodiments, the unmanned vehicle 120 mayinclude a wind sensor that is configured to gather wind data (e.g.,operational data). In some embodiments, the wind data includes windspeed and/or wind direction. Such wind data may be useful fordetermining wind conditions and/or trends in wind conditions for sailingat remote locations, which may help a user determine whether or not totravel to the location. In some embodiments, the unmanned vehicle 120may account for its own current speed and/or direction when measuringwind data (e.g., account for motion of the unmanned vehicle 120). Forexample, if the wind is 0 knots and the unmanned vehicle 120 istravelling at 10 knots, the unmanned vehicle 120 may take the 10 knotsoff the measured wind speed (e.g., 10 knots) to get the actual windspeed (e.g., 0 knots). This example ignores wind direction and windangle for simplicity of explanation—however, embodiments of the presentinvention may account for such factors when determining the wind data.

In some embodiments, the gathered wind data may be provided to themarine electronic device 105, such as for display to a user. In someembodiments, the wind data could be associated with a location of theunmanned vehicle 120 when it was acquired. Such information could beutilized on a display (such as described herein with respect to sonarand radar data). Additionally or alternatively, the wind data could beutilized in other ways (e.g., combining with other operational data,etc.).

In some embodiments, the marine electronic device 105 may be configuredto control/operate the unmanned vehicle 120 to have it travel to certainlocations or altitudes to gather wind data. In some embodiments, theunmanned vehicle 120 may be instructed to travel according to a surveypattern to gather wind data to provide back to the marine electronicdevice 105. Such wind data, in some embodiments, may be collected andformed into a map and/or displayed with respect to the marine vessel tohelp an operator determine where to travel (such as to maximize windconditions for easy travel).

In some embodiments, the unmanned vehicle 120 may include a heave sensorthat is configured to gather heave/roll/pitch data (e.g., operationaldata). As used herein heave/roll/pitch data may be referred to as “heavedata”. Such heave data may be useful for determining wave conditions atvarious locations.

Depending on the configuration of the heave sensor and/or the positionof the heave sensor on the unmanned vehicle 120, the unmanned vehicle120 may be configured to land on the surface of the water to gather theheave data. For example, FIG. 12 illustrates an example marineenvironment 501 where the unmanned vehicle 20 has landed on a wavysurface 12′ of the water. The unmanned vehicle 20 includes a heavesensor 580 that is used to measure the heave/roll/pitch of the waves asthe unmanned vehicle 20 floats on the water surface. In someembodiments, the heave sensor may comprise an IR laser that couldmeasure the height of the surface of the water while the unmannedvehicle 120 hovers above the water surface. In this regard, embodimentsof the present invention contemplate different types of heave sensorsfor determining the heave/roll/pitch of the water surface.

In some embodiments, the gathered heave data may be provided to themarine electronic device 105, such as for display to a user. In someembodiments, the heave data could be associated with a location of theunmanned vehicle 120 when it was acquired. Such information could beutilized on a display (such as described herein with respect to sonarand radar data). Additionally or alternatively, the heave data could beutilized in other ways (e.g., combining with other operational data,etc.).

In some embodiments, the marine electronic device 105 may be configuredto control/operate the unmanned vehicle 120 to have it travel to certainlocations to gather heave data. In some embodiments, the unmannedvehicle 120 may be instructed to travel according to a survey pattern togather heave data for providing back to the marine electronic device105. Such heave data, in some embodiments, may be collected and formedinto a map and/or displayed with respect to the marine vessel to help anoperator determine where to travel (such as for minimal wave activityfor easy travel).

In some embodiments, the unmanned vehicle 120 may include an AutomaticIdentification System (AIS) that is configured to gather informationregarding other marine vessels and/or objects (e.g., operational data).Additionally, the unmanned vehicle 120 may provide its location throughthe AIS to other marine vessels. In this way, the unmanned vehicle 120may become known to other marine vessels (i.e., marine vessels otherthan the marine vessel associated with the marine electronic device 105that may control/operate the unmanned vehicle 120).

In some embodiments, the gathered AIS data may be provided to the marineelectronic device 105, such as for display to a user. Additionally oralternatively, the AIS data could be utilized in other ways (e.g.,combining with other operational data, etc.). For example, whenperforming tasks, the AIS may provide the location of the unmannedvehicle 120 to other marine vessels. Such an embodiment may be useful insearch and rescue operations.

In some embodiments, the marine electronic device 105 may be configuredto control/operate the unmanned vehicle 120 to have it travel to certainlocations to gather AIS data. For example, the marine vessel may beunable to gather AIS data from its own AIS and, thus, may instruct theunmanned vehicle 120 to travel to a location to gather AIS data that itprovides to the marine electronic device 105. Such AIS data could beused to update a chart or map showing the AIS data. In some embodiments,the unmanned vehicle 120 may be configured to extend the range oftransmission or reception of AIS data or other transmission/receptiondata (such as VHF voice communications). For example, the unmannedvehicle 120 may travel to a high altitude to re-transmit the data inorder to extend the transmission range of VHF, such as by extending theline-of-sight. In some embodiments, the unmanned vehicle 120 may travelto the edge of range of transmission of the VHF of the marine vessel(e.g., 9 miles) and may gather and retransmit from that location inorder to extend the range of the VHF for the marine vessel. Likewise,the unmanned vehicle 120 may collect any type of data and travel towithin range of a remote VHF receiver (e.g., on land) in order totransmit the collected data.

Tracking Operation

In some embodiments, the unmanned vehicle 120 may be configured to trackone or more targets (e.g., fish, marine vessels, other objects, etc.).Depending on the configuration, the unmanned vehicle 120 may track thetarget based on its own systems and sensors and/or in response toinstructions provided by the marine electronic device 105 (e.g., throughcontrol signals). In such a regard, the unmanned vehicle 120 can be usedto track targets for users of the marine vessel, providing informationor other data about the target, such as its location, shape, video ofthe target, etc.

In some embodiments, the marine electronic device 105 may be configuredto provide instructions to the unmanned vehicle 120 to cause it tofollow and or intercept a target. In such embodiments, the marine vessel(associated with the marine electronic device 105) may be further awayor unavailable for tracking the target. In this regard, in someembodiments, the marine electronic device 105 can determine the locationor future location of the target, such as based on the current location,speed, altitude, and/or direction of travel of the target. Suchinformation could be gathered from sensors or systems (e.g., from radardata, sonar data, AIS data, camera data, etc.) that communicate with themarine electronic device 105 including, for example, from the unmannedvehicle 120. Then, based on the location or future location of thetarget, the marine electronic device 105 may be configured to determinea desired location for the unmanned vehicle 120 corresponding to thetarget. The desired location (or a set of desired locations) can then beprovided via instructions to the unmanned vehicle 120. In response, theunmanned vehicle 120 can travel accordingly and “track” the target.

FIG. 13A illustrates an example marine environment 601 that includes afish 613 swimming near the surface of the water. In the depictedembodiment, the unmanned vehicle 20 includes a camera 630 that isoriented toward the fish 613. In some embodiments, the unmanned vehicle20 may be tracking the fish 613.

The targets may be generated in any number of ways. For example, in someembodiments, a user may select an object visible in sonar data to be thetarget (e.g., in 3D sonar views). In another example, a user may selectanother marine vessel, such as from AIS data. A further example includesa user selecting the target from radar data, such as using an auto radarplotting aid (ARPA). In yet another example, digital selective calling(DSC) can be utilized to select a target. Further, other means can beapplied to select a target, such as using object recognition from cameradata. Notably, the above types of data (and others) can be acquiredusing any systems or sensors, including systems and sensors of theunmanned vehicle 120.

FIG. 13B illustrates an example marine electronic device 40 with ascreen 45 that is displaying an image 600 of a fish 613 in a body ofwater 611 from the camera of the unmanned vehicle (e.g., such as what isillustrated in FIG. 13A). In the depicted embodiment of FIG. 13B, theuser 70 is selecting the fish 613 via a touchscreen selection with theirfinger 71. In response, a pop-up window 623 with the question “TRACK?”is displayed for possible selection by the user 70. In this regard, theuser may provide user input selecting a target (e.g., the fish 613) andan indication that the user would like the fish 613 to be tracked. Inresponse, the marine electronic device 105 may determine instructions toprovide to the unmanned vehicle 120 to cause it to track the fish 613.

In some embodiments, object recognition technology could be used todifferentiate the fish 613 within the camera generated image. Further,depending on the configuration, the unmanned vehicle 120 and/or themarine electronic device 105 (through the camera/camera image from theunmanned vehicle 120) may be configured to utilize object recognition tocontinually discern the fish 613 to enable tracking by the unmannedvehicle 120. Additionally or alternatively, the marine electronic device105 may provide instructions to the user (e.g., navigation instructions)on how to direct the marine vessel or unmanned vehicle 120 so as totrack the target fish.

Though the above described examples detail tracking a fish, otherobjects are contemplated by embodiments of the present invention (e.g.,marine vessels, other unmanned vehicles, buoys, debris, bait schools,etc.).

FIG. 13C illustrates another way to select a target to track using radardata. In particular, FIG. 13C illustrates an example marine electronicdevice 40 with a screen 45 that is displaying a radar image 600′ thatincludes an object 633 within the coverage area 631 around the centerpoint 620 (e.g., the marine vessel or the unmanned vehicle). In thedepicted embodiment, the user 70 is selecting the object 633 via atouchscreen selection with their finger 71. In response, a pop-up window623 with the question “TRACK?” is displayed for possible selection bythe user 70. In this regard, the user may provide user input selecting atarget (e.g., the object 633) and an indication that the user would likethe object 633 to be tracked. In response, the marine electronic device105 may determine instructions to provide to the unmanned vehicle 120 tocause it to track the object 633.

FIG. 13D illustrates an example marine electronic device 40 with ascreen 45 that is displaying a chart image 600″ of the body of water 605that includes a representation of the marine vessel 610, unmannedvehicle 620, and a possible target 643. In some embodiments, theinformation gathered from the various systems or sensors of the marineelectronic device 105 and/or unmanned vehicle 120 may be used todetermine the location, the type of target, the shape of the target,among other things—which can be used to create a representation of thetarget that can be displayed at an appropriate position on the chart. Insome embodiments, the user can then select the target and select anoption to track the target accordingly.

In some embodiments, the unmanned vehicle 120 (such as through controlby the marine electronic device 105) may be configured to track one ormore fishing lures. In this regard, object recognition and/or the knownlocation of the lures may be utilized to enable tracking to occur.Further, the unmanned vehicle 120 may be configured to point/orient thecamera to where the lure is to enable a remote user to watch forpossible fish activity (e.g., bites). In some embodiments, objectrecognition technology could be used to provide an alert to the userwhen fish activity is determined (such as based on streaming cameradata). Such embodiments can be useful for game fishing where multiplelures are dragged behind the marine vessel and users may want to bealerted to fishing activity on the lures to catch the fish.

As will be apparent from embodiments described herein, in someembodiments, the marine electronic device 105 may control and/or operatea plurality of unmanned vehicles 120. In such embodiments, the marineelectronic device 105 may be configured to determine separateinstructions for each unmanned vehicle 120 such that they arecoordinated to track the target together. Such a scenario may be usefulwhen the movement of the target is unpredictable, maintaining “eyes on”the target using multiple unmanned vehicles.

Along similar lines, in some embodiments, the marine electronic device105 may be configured to utilize information from a plurality ofunmanned vehicles 120 to determine location data and/or other data(e.g., speed, direction of travel, size, etc.) about an object. Forexample, by positioning multiple unmanned vehicles 120 in a coordinatedeffort about an object, data from the various systems of the unmannedvehicles 120 can be utilized to determine and/or gather informationabout the object. For example, sonar data, radar data, camera data,orientation data, among others can be used from multiple sources aroundan object to determine characteristics of the object. Notably, themarine electronic device 105 may, in some embodiments, need to accountfor the orientation and relative position of the unmanned vehicles 120with respect to each other and/or the object when determiningcharacteristics regarding the object. In some embodiments, datacollected by the marine electronic device 105 systems or sensors (orfrom systems or sensors from other remote devices) may be used inaddition to or in the alternative of multiple unmanned vehicles 120.Notably, the marine electronic device 105 may, in some embodiments, needto account for the orientation and relative position of the unmannedvehicles 120 with respect to those systems and sensors when determiningcharacteristics regarding the object.

FIG. 14 illustrates an example marine environment 601′ that shows a fish613 and two unmanned vehicles 20 a, 20 b. The first unmanned vehicle 20a is flying to the right of the fish 613 and a camera 630 a is orientedtoward the fish 613, wherein the camera 630 a has a coverage areaindicated by 631 a. The second unmanned vehicle 20 b is flying behindthe fish 613 and a camera 630 b is oriented toward the fish 613, whereinthe camera 630 b has a coverage area indicated by 631 b. Using the knownlocations of each unmanned vehicle 20 a, 20 b and the known orientationsof the cameras 630 a, 630 b, the marine electronic device may beconfigured to determine characteristics and/or other information (e.g.,location, size, speed, direction of travel, etc.) regarding the fish613.

Though the embodiment shown in FIG. 14 utilizes cameras, other systemsof the unmanned vehicle may be used (e.g., sonar, radar, etc.). Suchsystems can, in some cases, be used in conjunction (e.g., data from theradars and the cameras can be used together) to determinecharacteristics and/or other information about the object.

Search and Rescue Operations

In some embodiments, the unmanned vehicle 120 may be configured toperform search and rescue operations. In some embodiments, the marineelectronic device 105 may be configured to control and/or operate theunmanned vehicle 120 (or a plurality of unmanned vehicles 120) toperform search and rescue operations. In this regard, unmanned vehicles120 may provide a useful means to efficiently and effectively aid insearch and rescue. Some examples of how the unmanned vehicles 120 mayassist in search and rescue operations are described below, although thebelow examples are not meant to be limiting as there are many differentsearch and rescue tasks capable of being performed by the unmannedvehicles 120 described herein.

In some embodiments, the unmanned vehicle 120 may be configured tosearch an area, such as for stranded persons, persons' overboard,missing vessels, etc. The unmanned vehicle 120 may travel, for example,according to pre-set search patterns and/or under the direction of auser/marine electronic device 105. Some example search patterns includea creeping line ahead search, a sector search, an expanding box search,following a shoreline, following a track, following a depth contour,etc.

Depending on the configuration, the unmanned vehicle 120 may travelaccording to instructions provided by the marine electronics device 105(such as from one or more control signals). In response, the unmannedvehicle 120 may cause the propulsion system to cause the unmannedvehicle to travel accordingly.

Such embodiments may be particularly useful for covering additionalsearch area (e.g., the marine vessel covers one search area, while theunmanned vehicle covers another). Further, such embodiments may beuseful when the marine environment being searched is hazardous for themarine vessel, such as due to weather, shallow water, searching beyondthe surf or reef, etc. Indeed, when searching, there may not be enoughinformation or time to scout out the search area. Using an unmannedvehicle 120 may help alleviate some of these concerns. In someembodiments, the unmanned vehicle 120 may scout ahead of the marinevessel so as to provide an indication of any upcoming hazards for themarine vessel.

In some embodiments, the marine electronic device 105 may be configuredto update the search pattern based on received operational data from theunmanned vehicle 120 and/or other data from other sources (e.g., otherunmanned vehicles, authorities, other marine vessels, systems and/orsensors on the marine vessel, etc.). For example, the marine electronicdevice 105 may receive information from an external source indicatingthat the wind conditions, tide conditions, or other factors havechanged. In response, the marine electronic device 105 may determine anupdated search pattern or area for the unmanned vehicle 120 (such as toaccount for the changes).

In some embodiments, the marine electronic device 105 may control and/oroperate a plurality of unmanned vehicles 120. In such embodiments, themarine electronic device 105 may provide separate instructions to eachunmanned vehicle 120 (or each team of unmanned vehicles 120) tocoordinate their movements or operations for performing an efficientsearch and rescue operation.

In some embodiments, the unmanned vehicle 120 (such as from instructionsfrom the marine electronic device 105) may be configured to gatheroperational data while searching. Such operational data may be providedback to the marine electronic device 105 (or other marine vessels, theauthorities, Coast Guard, etc.), such as in real-time, to assist insearch and rescue. For example, the unmanned vehicle 120 may use acamera 133 to capture camera data of the searched area. The camera datacould be provided back to the marine electronic device 105 for displayand/or use. In some embodiments, object recognition technology could beused at the marine electronic device 105 and/or the unmanned vehicle 120to locate the desired object of interest within the camera data.Further, tracking could be enabled once the desired object of interestis located.

Though the above examples describe providing camera data as feedback,other types of operational data are contemplated, such as sonar data,radar data, IR sensor data (such as described above in the OtherSensor(s)/System(s) Operation section), other system/sensor data, etc.

Along these lines, in some embodiments, the unmanned vehicle 120 mayutilize radar (e.g., radar system 137) or an AIS to find marine vesselsor objects, such as AIS targets or radar search and rescue transponders(SARTs). For example, a coast guard vessel or other marine vessel(including the marine vessel associated with the unmanned vehicle 120)may be unable to see the target, such as due to high waves, lack ofheight for an effective radar reading, other obstructions, among otherthings. In such circumstances, the unmanned vehicle 120 may employ itsown radar or AIS (or other sensor/system) to locate and/or travel to atarget. Accordingly, in some embodiments, the unmanned vehicle 120 maysimply provide location information about the target to the marineelectronic device 105 so that the marine vessel can travel to thetarget.

Though the above embodiments describe providing operational dataremotely, in some embodiments, the unmanned vehicle 120 may beconfigured to perform the search operation and report back to the marineelectronic device 105 or marine vessel for uploading of the operationaldata. In such embodiments, the upload may be performed using a shortrange wireless connection or through manual upload (e.g., a universalserial bus (USB)).

In some embodiments, the unmanned vehicle 120 may include equipmentand/or systems that it may use during the search and rescue operation.In this regard, in some embodiments, the unmanned vehicle 120 mayinclude a light system 138 that can be utilized to aid in search andrescue operations. For example, the unmanned vehicle 120 may fly asearch pattern with the light shining over the water to help locate aperson overboard. In some embodiments, the unmanned vehicle 120 maysearch for a person overboard (such as described herein using, forexample, object recognition technology and/or IR sensors). Upon locatinga person overboard, the unmanned vehicle 120 may be configured to hoverover the person and shine the light 138 over the person in the water.For example, FIG. 15 illustrates a marine environment 701 and an examplesituation where the unmanned vehicle 20 has located a person 717floating in the water 11. The unmanned vehicle 20 has shined a lightbeam 786 over the person 717 using the light system 785.

In some embodiments, the unmanned vehicle 120 may be configured to (suchas through instruction by the marine electronic device 105) perform aman overboard operation that includes launching from the marine vesseland searching for the man overboard. In some embodiments, when the manis located, the unmanned vehicle 120 may shine a light 138 onto the manfor easy spotting by the marine vessel crew. In some embodiments, theman overboard operation could be performed based on a selection via theuser interface of the marine electronic device 105. Alternatively, theman overboard operation could be performed automatically, such as inresponse to a man overboard alarm being sounded or by the marineelectronic device 105 determining that a crew member has fallenoverboard. The latter could be determined in the situation where thecrew member is wearing a tracking device (such as on their uniform).Further, such a tracking device could be used to quickly locate the crewmember, such as to enable the unmanned vehicle 120 to fly to andinteract with the overboard crew member. For example, the unmannedvehicle 120 may shine a light on the crew member, deploy equipment(e.g., VHF radio, floatation device, etc.), deploy a beacon, land on thewater as a buoyancy aid, emit audible sounds to draw attention to thelocation, initiate a strobe light, begin 2 way communication with theoverboard crew member, among many other search and rescue operations. Insome embodiments, the unmanned vehicle 120 may hover over the overboardcrew member to provide a visual indicator of the location of the crewmember, which may be helpful in avoiding the crew member in the waterand/or traveling to the crew member.

In some embodiments, the unmanned vehicle 120 may be in communicationwith the marine electronic device 105 for navigation, such as tonavigate to or mark the location of the overboard crew member. In someembodiments, routes or navigation instructions to the overboard crewmember could be automatically generated. Further, the autopilot could beautomatically engaged or the marine vessel could be automaticallystopped.

In some embodiments, the unmanned vehicle 120 may be configured todeploy equipment (e.g., in response to instructions from a marineelectronic device 105), such as may be useful during search and rescueoperations. For example, in the search and rescue operation context, insome embodiments, the unmanned vehicle 120 may be configured to travelto a person in the water. When there, the unmanned vehicle 120 may hoverover the person and deploy equipment that may aid the person. Forexample, the unmanned vehicle 120 may deploy (e.g., drop, lower, etc.) aflotation device, a rope, a radio, a transponder, a beacon, among otherthings. In some embodiments, the flotation device could inflate as it isreleased from the unmanned vehicle 120. FIG. 16 illustrates a marineenvironment 701′ and an example situation where the unmanned vehicle 20has located a person 717 floating in the water 11. The unmanned vehicle20 has deployed a rope 796 over the person 717 from an equipment housing795.

Task Operations

As detailed herein, in some embodiments, the unmanned vehicle 120 mayinclude systems, equipment, storage for holding or transporting things,and/or many other features. In this regard, embodiments of the presentinvention contemplate many different uses for example unmanned vehicles120 described herein, such as various tasks or operations describedherein. Such tasks or operations, in some embodiments, may be performedvia control/operation by the marine electronic device 105. In thisregard, as with many embodiments described herein, the features andcapabilities of the marine electronic device 105 may be useful inenabling the unmanned vehicle 120 to perform the desired task oroperation. The following examples are not meant to be limiting, as manydifferent tasks or operations are contemplated as part of the presentinvention.

In some embodiments, the unmanned vehicle 120 may be configured tomeasure or map the distance to a point of interest. For example, in someembodiments, the unmanned vehicle 120 may be configured to travel to aposition or a location (e.g., a point of interest), such as undercontrol by a user (e.g., through the marine electronic device 105). Oncethere or along the way, the unmanned vehicle 120 may be configured tosend location data back to the marine electronic device 105. The marineelectronic device 105 may then determine the location coordinates and/ordetermine the distance to and/or bearing to the position/location of thepoint of interest. Such an embodiment may be useful when the point ofinterest is unmarked. Additionally, the unmanned vehicle 120 may travelas the marine vessel would (e.g., around rocks or by depth contour) andmeasure the actual distance of travel for the marine vessel. Suchmeasurements could be useful for determining a projected time of travelfor the marine vessel and/or if extra fuel would be necessary.

In some embodiments, the unmanned vehicle 120 may be configured todeploy bait or chum at a desired location, such as may be useful forfishing. In this regard, the unmanned vehicle 120 may be configured(such as based on instructions from a marine electronic device 105) totravel to a location and deploy the bait or chum. Such an action mayinclude hovering over a position and dropping the bait or chum into thewater. In some embodiments, the unmanned vehicle 120 may be configuredto perform such a task in response to the marine electronic device 105receiving user input indicating a desire to perform the task (e.g., auser selects a location, such as a waypoint, or a distance/position withrespect to the marine vessel and then selects “Deploy Bait/Chum”). Suchembodiments are useful, as normally a fisherman would have to move themarine vessel to the location to drop in the bait or chum Then thefisherman may have to move the boat to begin fishing. In someembodiments, other objects could be deployed, such as a buoy, dynamite,a fish lure, among other things.

In some embodiments, the unmanned vehicle 120 may be configured to actas a fish attractor. In such embodiments, the unmanned vehicle 120 (suchas under instruction by the marine electronic device 105 and/or theuser) may be configured to travel to a position (such as a futurefishing spot). Once there, the unmanned vehicle 120 may emit fishattracting sounds into the underwater environment. This could beaccomplished by landing on the water surface and emitting sound into theunderwater environment via one or more speakers. Additionally oralternatively, the unmanned vehicle 120 may be configured to land on thewater surface or hover over the water surface and deploy or drop a soundemitting device into the water. In some embodiments, the unmannedvehicle 120 may be configured to perform this task in response toreceiving user input to a marine electronic device 105 (e.g., a userselects a location, such as a waypoint, or a distance/position withrespect to the marine vessel and then selects “Attract Fish”).

In some embodiments, the unmanned vehicle 120 may be configured to actas an out-rigger. In this regard, the unmanned vehicle 120 may carry aline (e.g., rope) to a position or location. In some embodiments, oneend of the line may be attached (e.g., to the marine vessel) or held bya person onboard of the marine vessel such that the unmanned vehicle 120is used to carry the other end of the line to the desired position(e.g., another marine vessel, a person on a dock, etc.). In someembodiments, the unmanned vehicle 120 may be configured to carry theline such that it wraps around a remote object (e.g., a buoy, a pillar,another marine vessel, etc.). In such embodiments, the unmanned vehicle120 may travel according to instructions provided by the marineelectronic device 105 and/or from instructions based on direct controlfrom a user. In some embodiments, when defining the location for theunmanned vehicle 120 to travel, the marine electronic device 105 maydetermine a position relative to the marine vessel. Along these lines, auser may input a position relative to the marine vessel (e.g., 5 feet toport), and the marine electronic device 105 may convert the position tolocation coordinates that are geo-referenced for instructing theunmanned vehicle 120.

In some embodiments, the unmanned vehicle 120 may be configured toenable transfer of objects, such as from marine vessel to marine vessel,marine vessel to land, etc. For example, as detailed herein, theunmanned vehicle 120 may include a storage compartment or an attachmentmeans for holding objects. In such embodiments, a user may place orattach objects (e.g., a phone, beer, a radio, a rope, lunch, etc.) tothe unmanned vehicle 120. Then, whether under control of the marineelectronic device 105 and/or direct control by a user, the unmannedvehicle 120 may be configured to travel to a position and land or hoverto enable another user to acquire the object. In some embodiments, theunmanned vehicle 120 may deploy or drop the object on its own.

In some embodiments, the unmanned vehicle 120 may be configured to aidin the search for objects in the water. For example, the unmannedvehicle 120 may be configured to act in a water ski search mode, such asfor locating a dropped water ski (e.g., when someone drops from 2 waterskis to 1 water ski). In this regard, the unmanned vehicle 120 could useobject recognition technology or be under control by a user whileproviding streaming video back to the user to locate the object in thewater for retrieval. The location could be marked at the marineelectronic device 105 and/or the unmanned vehicle 120 may stay at thelocation providing a visual indicator of the location of the object.Though the above example describes a water ski, other objects are alsocontemplated (a lost hat, shoes, flotation device, etc.).

In some embodiments, the unmanned vehicle 120 may be configured to actas a spy, such as for spying on other fisherman during a competition orother sailors during a sailing race. In this regard, in someembodiments, the unmanned vehicle 120 may be configured to followanother marine vessel to gather intelligence, such as providingstreaming video, gathering wind data, gathering speed data, gatheringdirection or travel data, gathering location data, gathering other data,etc. This intelligence could be provided back to the marine electronicdevice 105 such as for display to a user.

In some embodiments, the unmanned vehicle 120 may act as a wirelessrange or VHF range extender. For example, the unmanned vehicle 120 mayhave a wireless transceiver or VHF transceiver. In such embodiments, theunmanned vehicle 120 (such as from instruction from the marineelectronic device 105) may be configured to travel to a position remotefrom the marine vessel (and/or the marine electronic device 105) and actas a range extender. In this regard, in some embodiments, the higher theantenna, the better the range. As such, the unmanned vehicle 120 may beconfigured to fly to a high altitude to achieve a longer range. In someembodiments, the unmanned vehicle 120 may be configured to act as a WiFirange extender, such as to extend WiFi capabilities to or from themarine vessel (or another marine vessel). In this regard, the marinevessel (or a person on another marine vessel or raft) may be too faraway from the WiFi source. In such an embodiment, the unmanned vehicle120 may travel to a point in between the user and the source of the WiFiand act as a range extender to enable connectivity for the user.

In some embodiments, the unmanned vehicle 120 may utilize wirelesscapabilities to gather information. For example, the marine vessel maybe out of range of wireless connection to an external network (e.g.,cellular, satellite, WiFi, etc.). In such embodiments, the unmannedvehicle 120 (such as through instruction from the marine electronicdevice 105) may be configured to travel to a position in which it isable to connect to an external network to download desired information(e.g., weather data, news, software updates for various systems on themarine vessel, etc.). Then, the unmanned vehicle 120 may transmit thatinformation back to the marine electronic device 105 and/or return tothe marine vessel for uploading of the information. Likewise, theunmanned vehicle 120 may include stored data from the marine electronicdevice 105 (such as sonar logs, emails for sending, etc.) that can thenbe transmitted using the discovered external network—thereby enabling auser of the marine electronic device 105 to use the unmanned vehicle 120to transmit data to an external network that may be otherwiseinaccessible at the location of the marine electronic device 105.

In some embodiments, the unmanned vehicle 120 may be configured todetect available internet connections. For example, the unmanned vehicle120 (such as by instruction from the marine electronic device 105) maybe configured to search for available internet connectivity (e.g.,cellular, WiFi, satellite, etc.). In some embodiments, the unmannedvehicle 120 may determine the locations in which internet connectivityis possible, and provide those locations back to the marine electronicdevice 105. In some embodiments, a user may pick which type ofconnectivity is desirable (e.g., cellular, WiFi, satellite, etc.).Further, in some embodiments, there may be a distinction between static(e.g., land-based) connections and mobile connections (e.g., WiFi fromanother marine vessel).

In some embodiments, the marine electronic device 105 may be configuredto “map” out available internet connectivity, such as on a chart of thebody of water. Further, the type of connectivity could be indicated. Insuch a regard, a user could be aware of where the marine vessel (orunmanned vehicle 120) could travel on the body of water (or land for theunmanned vehicle) to establish internet connectivity. FIG. 17illustrates an example chart image 800 on a screen 45 of a marineelectronic device 40 that shows the location of the marine vessel 810and unmanned vehicle 820 on the body of water. Additionally, variousareas 892 of the body of water that have internet connectivity areshaded for easy reference by a user. As such, the user may travel towithin the shaded areas 892 to enable internet connectivity. Such anembodiment would be useful for a marine environment where internetconnectivity is often very limited. Indeed, in some embodiments, theunmanned vehicle 120 may be controllable in certain internet finding orassessing modes. For example, a user could employ a “Go Find Internet”mode that causes the unmanned vehicle 120 to travel around to findviable internet hotspots. These hotspots could be mapped or communicatedback to the marine vessel. Additionally, assessment could be made as tovarious internet performance factors for each hotspot—such as is itfixed or mobile, the strength of signal, the likelihood the hotspot willstill be available if the marine vessel/unmanned vehicle travels to thelocation, is it password protected, etc. This assessment could becommunicated back (e.g., to the marine electronic device 105) to enablea user to decide whether to travel or otherwise use the discoveredhotspot. In some embodiments, the assessment could be made after aninternet “map” of the surrounding location is acquired (such as inresponse to a user requesting the assessment). The assessment could bemade by the unmanned vehicle 120 and/or the marine electronic device105. Further, in some embodiments, the unmanned vehicle 120 or themarine electronic device 105 may be configured to determine that themarine vessel or the unmanned vehicle 120 is unable to travel to anyinternet hotspot or potential internet hotspot location (e.g., due tofuel or battery life constraints, weather, etc.). This information couldbe communicated to the user of the marine electronic device 105 and/orunmanned vehicle 120 in any form (e.g., an alarm, a message such as“Error: Drone Unable to Reach Requested Location”, other notification,etc.).

In some embodiments, the unmanned vehicle 120 may be configured to actas a “buoy”, such as may be useful for fishing activity. For example, afisherman may determine that there is desirable structure at a specificlocation under the water (e.g., fish attracting structure). However, thefisherman may not be able to visibly discern the location. In such asituation, the unmanned vehicle 120 may be positioned at (or travel to)the location and float at or hover above the location—thereby visuallymarking the location (and the structure). In this manner, even if thefisherman drifts or moves his/her eyes, the unmanned vehicle 120 (actingas a “buoy”) will enable the fisherman to easily locate the desiredstructure.

Depending on the desired configuration and/or desired complexity, theunmanned vehicle 120 may be configured as a self-contained unit to actas a “buoy” such that it is not controlled (or even in communicationwith the marine electronic device 105). Such embodiments may enable theunmanned vehicle 120 to be inexpensive and more cost effective.Alternatively, in some embodiments, the unmanned vehicle 120 may beconfigured to communicate with and/or be controlled or operated by themarine electronic device 105. In such embodiments, the unmanned vehicle120 may enter a “Buoy Mode”, such as in response to user input to themarine electronic device 105.

In some embodiments, the unmanned vehicle 120 may be small in size andeasy to pick up and drop into the water at a desired location. In suchembodiments, the unmanned vehicle 120 may be handheld or pocket-sized.In this regard, a user may simply place the unmanned vehicle 120 in thewater at the location and it will begin acting as a buoy. Alternatively,the unmanned vehicle 120 may be instructed (such as from the marineelectronic device 105) to travel to a certain location and remain there(either floating on the water surface or hovering above). Accordingly,the unmanned vehicle 120 may cause its propulsion system to cause it totravel to the location accordingly.

In some embodiments, the unmanned vehicle 120 may be configured tovirtually anchor in the desired location. For example, the unmannedvehicle 120 may include a location sensor (e.g., GPS system) that candetermine the location of the unmanned vehicle 120. Additionally, theunmanned vehicle 120 may include a propulsion system that can be used tokeep the unmanned vehicle 120 “anchored” at the desired location. Inthis regard, as the unmanned vehicle 120 drifts (e.g., by wind orwaves), the location sensor may determine that the unmanned vehicle 120is no longer at the desired location and, in response, the propulsionsystem can be engaged to return the unmanned vehicle 120 to the desiredlocation.

In some embodiments, the unmanned vehicle 120 may include one or morelights. For example, the unmanned vehicle 120 may include a light beaconthat is configured to emit light (e.g., an LED) to the surroundingenvironment (e.g., above water) to provide a visual indication of wherethe unmanned vehicle 120 is. This may enable night fishing and/or aid innavigation and pick up of the unmanned vehicle 120. In some embodiments,the unmanned vehicle 120 may include one or more underwater lights thatare positioned to enable emitting light to the underwater environment.This may be useful for divers in searching wreckage or for fishattracting.

In some embodiments, the unmanned vehicle 120 may be configured tocommunicate with one or more remote devices (e.g., the marine electronicdevice 105 and/or an external network). In such embodiments, theunmanned vehicle 120 may be configured to transmit its location (orgathered location data) to the remote device to aid in marking of thelocation and/or pick up of the unmanned vehicle 120. In response, themarine electronic device 105 may be configured to automaticallydetermine navigation aids to travel to the unmanned vehicle 120. Furtheran autopilot could be engaged to automatically travel to the location.

In some embodiments, other data may be transmitted to the remote device(such as described in the various embodiments herein) from the unmannedvehicle 120. For example, the unmanned vehicle 120 may include a sonarsystem that can gather sonar data from the underwater environment, suchas while the unmanned vehicle 120 floats on the water surface acting asa “buoy”. The sonar data could be transmitted back wirelessly (e.g.,through an external network, Bluetooth, etc.) and displayed for a user(such as on a screen of the marine electronic device 105). Such sonardata could be used to confirm that the desired structure or fish arestill present underneath the unmanned vehicle 120. In some embodiments,a user could upload the sonar data (or other operational data) manuallyonce the unmanned vehicle 120 is accessible (e.g., when it is back onthe marine vessel).

In some embodiments, the unmanned vehicle 120 may include a battery orother power source to operate its various components. In someembodiments, the unmanned vehicle 120 may include one or more solarpanels that can be used to directly power the components and/or rechargethe batteries.

FIG. 18 illustrates an example marine environment with an unmannedvehicle 20′ acting as a “buoy”. In the depicted embodiment, the unmannedvehicle 20′ is floating on the surface 12 of the body of water 11 near amarine vessel 10. In particular, the unmanned vehicle 20′ has beenpositioned so that it marks the location of a rock 14 on the floor 13 ofthe body of water 11. As shown, the unmanned vehicle 20′ includes ahousing 22′ that enables it to float on the water surface 12. Apropulsion system 25′ enables the unmanned vehicle 20′ to correct itsposition so that it is virtually anchored (as described above). Theunmanned vehicle 20′ also includes a light beacon 28′ for additionalvisual distinction to a user. Further, the unmanned vehicle 20′ includesa sonar system 30′ that can be used to gather sonar data from theunderwater environment, such as to determine if fish are present aroundthe rock 14.

The above example embodiments of using an unmanned vehicle 120 as a“buoy” provide many different advantages over regular buoys. Forexample, a regular buoy is unable to correct its location to account fordrift due to waves or wind. Often, such fishing buoys are anchored tothe bottom to avoid draft. However, in order to ensure the buoy stillremains near the location, the correct length of rope for the anchormust be determined. The present invention avoids the need to make thatdetermination. Further, there is no disturbing of fish by dropping ananchor into the water—as the present invention does not require ananchor to be dropped.

Example System Architecture

FIG. 19 shows a block diagram of an example system 100 capable for usewith several embodiments of the present invention. As shown, the system100 may include a number of different modules or components, each ofwhich may comprise any device or means embodied in either hardware,software, or a combination of hardware and software configured toperform one or more corresponding functions. For example, the system 100may include one or more unmanned vehicles 120, one or more marineelectronic devices 105, an external network 102, and one or more mobiledevices 109.

The system 100 may also include one or more communications modulesconfigured to communicate with one another in any of a number ofdifferent manners including, for example, via a network. In this regard,the communication interfaces (e.g., 113, 123) may include any of anumber of different communication backbones or frameworks including, forexample, Ethernet, the NMEA 2000 framework, GPS, cellular, WiFi, orother suitable networks. The network may also support other datasources, including GPS, autopilot, engine data, compass, radar, etc.Numerous other peripheral devices such as one or more wired or wirelessmulti-function displays (e.g., a marine electronic device 105) may beincluded in the system 100.

The marine electronic device 105 may include a processor 110, a memory112, a user interface 116, a display 114, one or more sensors 117 (e.g.,a location sensor, position sensor, heading sensor, orientation sensor(not shown), etc.), and a communication interface 113.

The processor 110 may be any means configured to execute variousprogrammed operations or instructions stored in a memory device such asa device or circuitry operating in accordance with software or otherwiseembodied in hardware or a combination of hardware and software (e.g., aprocessor operating under software control or the processor embodied asan application specific integrated circuit (ASIC) or field programmablegate array (FPGA) specifically configured to perform the operationsdescribed herein, or a combination thereof) thereby configuring thedevice or circuitry to perform the corresponding functions of theprocessor 110 as described herein. In this regard, the processor 110 maybe configured to analyze electrical signals communicated thereto toprovide processing to utilize any operational and/or location datadetected by the system 100 (e.g., operational data and/or location dataprovided by one or more unmanned vehicles 120).

In some embodiments, the processor 110 may be further configured toimplement signal processing or enhancement features to improve thedisplay characteristics or data or images, collect or process additionaldata, such as time, temperature, GPS information, waypoint designations,or others, or may filter extraneous data to better analyze the collecteddata. It may further implement notices and alarms, such as thosedetermined or adjusted by a user, to reflect depth, presence of fish,proximity of other watercraft, etc.

The memory 112 may be configured to store instructions, computer programcode, marine data, such as sonar data, chart data, location/positiondata, radar data, camera data, and other data associated with the sonarsystem (e.g., such as from the unmanned vehicle 120 or a navigationsystem operated by the marine electronic device) in a non-transitorycomputer readable medium for use, such as by the processor.

The communication interface 113 may be configured to enable connectionto external systems (e.g., an external network 102, one or more marineelectronic devices 105, one or more unmanned vehicles 120, and/or one ormore mobile devices 109). In this regard, the marine electronic device105 may include at least one transmitter configured to transmit, forexample, one or more signals according to example embodiments describedherein. Likewise, the marine electronic device 105 may include at leastone receiver configured to, for example, receive data from one or moreunmanned vehicles according to example embodiments described herein. Insome embodiments, the transmitter and receiver may be combined as atransceiver. Though the depicted embodiment shows specific examples ofexternal networks that the marine electronic device is configured tocommunicate with, there are many other contemplated systems (such as theoperation and/or control systems of the watercraft carrying the marineelectronic device).

The marine electronic device may also include one or more sensor(s),system(s), or other information/data collecting devices 117 (such as anyexample sensors or data collecting devices described in any embodimentsherein). For example, the marine electronic device 105 may include alocation sensor configured to determine the current location of themarine electronic device. Further, the marine electronic device mayinclude speed and/or direction detecting sensors that are configured todetermine the speed and/or heading of the marine electronic device (orthat of the watercraft/marine vessel associated with the marineelectronic device). Even further, the marine electronic device mayinclude a navigation system that is configured to enable navigationcapabilities (such as described herein in greater detail with respect tosome example embodiments). As will be apparent to one of ordinary skillin the art based on the disclosure herein, there are many differenttypes of sensors or data collection devices that can be utilized and/orintegrated within the marine electronic device.

The display 114 may be configured to display images and may include orotherwise be in communication with a user interface 116 configured toreceive input from a user. The display 114 may be, for example, aconventional LCD (liquid crystal display), a touch screen display,mobile device, or any other suitable display known in the art upon whichimages may be displayed.

In any of the embodiments, the display 114 may present one or more setsof marine data, such as operational data and/or location data of theunmanned vehicle, (or images generated from the one or more sets ofdata). Such marine data includes chart data, radar data, weather data,location data, position data, orientation data, sonar data, or any othertype of information relevant to the watercraft and/or unmanned vehicle.In some embodiments, the display may be configured to present suchmarine data simultaneously as in split-screen mode. In some embodiments,a user may select any of the possible combinations of the marine datafor display.

The user interface 116 may include, for example, a keyboard, keypad,function keys, mouse, scrolling device, input/output ports, touchscreen, or any other mechanism by which a user may interface with thesystem.

Although the display 114 of FIG. 19 is shown as being directly connectedto the processor 110 and within the marine electronic device 105, thedisplay 114 could alternatively be remote from the processor 110 and/ormarine electronic device 105. Likewise, in some embodiments, othercomponents of the marine electronic device 105 could be remotelylocated.

The unmanned vehicle 120 may include components (hardware or software)that are configured according to any of the example embodiments detailedherein in the same manner as similar components to those of the marineelectronic device 105. For example, the unmanned vehicle 120 may includea processor 121, a memory 122, and a communication interface 123.However, the corresponding processor 121, memory 122, and communicationinterface 123 may be configured according to example embodimentsdescribed herein with respect to example unmanned vehicles and theircorresponding tasks.

For example, the communication interface 123 may be configured to enableconnection to external systems (e.g., an external network 102, one ormore marine electronic devices 105, one or more unmanned vehicles 120,and/or one or more mobile devices 109). In this regard, the unmannedvehicle 120 may include at least one transmitter configured to transmit,for example, location and/or operational data according to exampleembodiments described herein. Likewise, the unmanned vehicle 120 mayinclude at least one receiver configured to, for example, receive one ormore control signals from one or more marine electronic devices 105according to example embodiments described herein. In some embodiments,the transmitter and receiver may be combined as a transceiver.

In addition to similar components described above with respect to amarine electronic device, the unmanned vehicles 120 may include one ormore additional components. For example, FIG. 19 depicts the additionalcomponents of a location sensor 128, a propulsion system 125, one ormore lights 138, deployable equipment 190, and other operationalcomponents 130 (including, for example, a sonar system 136, a radarsystem 137, a camera 133, and other sensor(s) 139).

The location sensor 128 may be configured to retrieve and/or detectlocation/position data corresponding to the current location of theunmanned vehicle. For example, the location sensor 128 may be a GPSenabled device.

The propulsion system 125 may include one or more motors configured topropel the unmanned vehicle 120. In this regard, depending on theconfiguration of the unmanned vehicle, the propulsion system 125 may beconfigured to propel the unmanned vehicle for flight (unmanned aerialvehicle), along the surface of the body of water (unmanned surfacevehicle), and/or under the water (unmanned submersible vehicle). In someembodiments, the propulsion system may be configured to perform anycombination of the three configurations, such that it may be configuredfor flight, surface travel, and/or submersible travel.

The one or more lights 138 may include, for example, any type of light.For example, the one or more lights 138 may include a spotlight that isconfigured to emit onto a spot in the water or a person according tosome example embodiments. Additionally or alternatively, the one or morelights 138 may be alert lights and/or indicators, such as for indicatinginformation (e.g., a location, a desired direction for the watercraft totravel (e.g., red/green buoy indicators), a message, etc.). In thisregard, different color lights and/or different frequency of operationof the lights can be used (e.g., Morse code, etc.).

The deployable equipment 190 may include any equipment that can be heldand deployed by the unmanned vehicle. Such equipment can be deployed andreleased (e.g., a radio, a floatation device, an oxygen tank, abeverage, etc.) or deployed and remain attached (e.g., a rope, a sonarsystem, other sensors, etc.). In some embodiments, the deployableequipment may be placed into a housing or attached to the unmannedvehicle 120.

The unmanned vehicle 120 may include one or more other operationalcomponents 130 (e.g., components that gather operational data). Forexample, the unmanned vehicle 120 may include a sonar system 136, aradar system 137, a camera 133, or other sensor(s)/system(s) 139.

The sonar system 136 may include a transducer assembly that is providedin one or more housings that provide for flexible mounting options withrespect to the watercraft. In this regard, for example, the housing maybe mounted onto a portion of the unmanned vehicle 120 or onto a deviceor component that may be attached to the unmanned vehicle 120 (e.g., arope, a cable, a shaft, or other component that is mountable relative tothe unmanned vehicle), including a bracket that is adjustable onmultiple axes, permitting omnidirectional movement of the housing.

The transducer assembly may include one or more transducers ortransducer elements positioned within the housing. Each transducer maybe configured as transmit/receive, transmit-only, or receive-only withrespect to transmitting one or more sonar beams and receiving sonarreturns. In some embodiments, each of the transducer elements may bepositioned within the housing so as to point toward a predetermined areaunder, to the side, or the front of the unmanned vehicle.

The shape of a transducer element may largely determine the type of beamthat is formed when that transducer element transmits a sonar pulse(e.g., a circular transducer element emits a cone-shaped beam, a lineartransducer emits a fan-shaped beam, etc.). Embodiments of the presentinvention are not limited to any particular shape transducer (or anyconfiguration—as it may include arrays, phased arrays, etc.). Likewise,transducer elements may comprise different types of materials that causedifferent sonar pulse properties upon transmission. For example, thetype of material may determine the strength of the sonar pulse.Additionally, the type of material may affect the sonar returns receivedby the transducer element. As such, embodiments of the present inventionare not meant to limit the shape or material of the transducer elements.Further, transducers may be configured to transmit and/or receive atdifferent frequencies. In this regard, embodiments of the presentinvention are not meant to be limited to certain frequencies.

Additionally, in some embodiments, the transducer assembly (or sonarsystem 136) may have a sonar signal processor and/or other componentspositioned within the housing. For example, one or more sonartransceivers (e.g., sonar transmitter/receiver), sonar transmitters,and/or sonar receivers may be positioned within the housing andconfigured to cause the one or more transducers to transmit sonar beamsand/or receive sonar returns from the one or more transducers. In someembodiments, the sonar signal processor, sonar transceiver, sonartransmitter, and/or sonar receiver may be positioned in a separatehousing.

The radar system 137 may be or include any radar components configuredto collect radar data. For example, the radar system may be configuredto transmit one or more radar waves and receive radar returns to formradar data. In this regard, the radar system may include one or moreantennas to transmit electromagnetic waves in the radio or microwavesdomain and capture any returns from objects in the path of the emittedsignal. The radar system may also include a receiver and/or a processorto process the radar data (e.g., form the radar data, determineproperties of any objects, etc.).

The camera 133 may be any type of camera, including a video camera. Insome embodiments, there may be multiple cameras oriented in different(or the same) directions. Additionally or alternatively, the camera 133may be configured to change direction (rotate, tilt, etc.). In suchembodiments, the camera 133 may be linked to an orientation and/ordirection sensor that can determine which direction the camera 133 isfacing. This may be a true direction (e.g., with respect to North) ormay be relative to the direction the unmanned vehicle 120 is facing (inwhich case the unmanned vehicle 120 may also have an orientation and/ordirection sensor to enable comparison and determination as to the actualdirection the camera is facing. In some embodiments, the camera 133 maybe configured to gather camera data, which may include streaming video,photographs, still photographs, panoramic photographs, etc.

The unmanned vehicle 120 may also include one or more sensor(s) or otherinformation/data collecting devices 139 (such as any example sensors ordata collecting devices described in any embodiments herein). Forexample, the unmanned vehicle 120 may include speed and/or directiondetecting sensors that are configured to determine the speed and/orheading of the unmanned vehicle. Additionally or alternatively, othersensors or data collection devices may be used (e.g., a position sensor,a heading sensor, an orientation sensor, an IR camera, a microphone, atemperature sensor, a wind sensor, a heave/roll sensor, an automaticidentification system (AIS), navigation system, among others). As willbe apparent to one of ordinary skill in the art based on the disclosureherein, there are many different types of sensors or data collectiondevices that can be utilized and/or integrated within the unmannedvehicle.

The system 100 may also include one or more mobile devices 109. Themobile devices 109 may include components known in the art for mobilephone or smartphones. In this regard, in some embodiments, the one ormore mobile devices 109 can be linked to and/or wirelessly controloperation of (or receive information from) one or more of the marineelectronic device 105 or the unmanned vehicle 120 according to anyembodiments described herein. For example, the mobile device 109 candisplay what is being displayed on the display 114 of the marineelectronic device 105—thereby enabling that information to be providedto the user no matter their location with respect to the marineelectronic device.

In some embodiments, the unmanned vehicle 120 may include a battery orother power source to enable power to the various components. In someembodiments, the power source may comprise at least one solar panelpositioned on the unmanned vehicle. In such embodiments, the solar panelmay be configured to charge one or more batteries, which may prolong thelife of the battery and/or operation time for the unmanned vehicle.

FIG. 20 shows a block diagram of an example environment 101 where system100 (or a similar system in accordance with embodiments describedherein) is capable for use. As shown, the environment 101 includes amarine electronic device 105 for a watercraft 103 (e.g., marine vessel).The marine electronic device 105 can be configured, such as according toembodiments described herein, to control operation of multiple unmannedvehicles (e.g., drone 1 (120) and drone 2 (120′)). Such control canoccur directly to the unmanned vehicles (e.g., through Bluetooth orother wireless control) or through an external network 102.

Example Flowchart(s) and Operations

Embodiments of the present invention provide methods, apparatuses andcomputer program products for controlling and operating one or moreunmanned vehicles in a marine environment. Various examples of theoperations performed in accordance with embodiments of the presentinvention will now be provided with reference to FIGS. 21-22.

FIG. 21 illustrates a control flow diagram according to an examplemethod for controlling navigation of an unmanned vehicle from a remotemarine electronic device according to an example embodiment 900. Theoperations illustrated in and described with respect to FIG. 21 may, forexample, be performed by, with the assistance of, and/or under thecontrol of one or more of the processor 121, memory 122, communicationinterface 123, location sensor 128, propulsion system 125, and/or othersensor(s)/system(s) 130 of the unmanned vehicle 120 (e.g., unmannedvehicle 920 of FIG. 21) and/or one or more of the processor 111, memory112, communication interface 113, user interface 116, display 114,and/or sensor(s)/system(s) 117 of the marine electronic device 105(e.g., marine electronic device 905 of FIG. 21).

Operation 950 may comprise retrieving (or determining) location dataindicative of a current location of the unmanned vehicle 920. Theprocessor 121, memory 122, communication interface 123, location sensor128, and/or other sensor(s)/system(s) 130 of the unmanned vehicle 920may, for example, provide means for performing operation 950. Operation952 may comprise transmitting the location data from the unmannedvehicle 920 and/or receiving the location data at the marine electronicdevice 905. The processor 121 and/or communication interface 123 of theunmanned vehicle 920 and/or the processor 111 and/or communicationinterface 113 of the marine electronic device 905 may, for example,provide means for performing operation 952.

Operation 954 may comprise determining the current location of theunmanned vehicle 920 at the marine electronic device 905 and based onthe received location data. The processor 111, memory 112, communicationinterface 113, and/or sensor(s)/system(s) 117 of the marine electronicdevice 905 may, for example, provide means for performing operation 954.Operation 956 may comprise determining the desired location for theunmanned vehicle 920 to travel to, wherein the determining is performedat the marine electronic device 905 and based on a planned route orwaypoint stored in the memory of the marine electronic device. Theprocessor 111, memory 112, user interface 116, display 114, and/orsensor(s)/system(s) 117 of the marine electronic device 905 may, forexample, provide means for performing operation 956. Operation 958 maycomprise transmitting a control signal from the marine electronic device905 and/or receiving the control signal at the unmanned vehicle 920,wherein the control signal includes instructions for the unmannedvehicle to travel to the desired location. The processor 111 and/orcommunication interface 113 of the marine electronic device 905 and/orthe processor 121 and/or communication interface 123 of the unmannedvehicle 920 may, for example, provide means for performing operation958.

Operation 960 may comprise causing the propulsion system of the unmannedvehicle to propel the unmanned vehicle to the desired location. Theprocessor 121, memory 122, communication interface 123, location sensor128, propulsion system 125, and/or other sensor(s)/system(s) 130 may,for example, provide means for performing operation 960.

FIG. 22 illustrates a control flow diagram according to an examplemethod for controlling operation of an unmanned vehicle from a remotemarine electronic device according to an example embodiment 970. Theoperations illustrated in and described with respect to FIG. 22 may, forexample, be performed by, with the assistance of, and/or under thecontrol of one or more of the processor 121, memory 122, communicationinterface 123, location sensor 128, propulsion system 125, and/or othersensor(s)/system(s) 130 of the unmanned vehicle 120 (e.g., unmannedvehicle 920 of FIG. 22) and/or one or more of the processor 111, memory112, communication interface 113, user interface 116, display 114,and/or sensor(s)/system(s) 117 of the marine electronic device 105(e.g., marine electronic device 905 of FIG. 22).

Operation 972 may comprise determining at the marine electronic device905 that operational data from the unmanned vehicle 920 is desired. Theprocessor 111, memory 112, communication interface 113, user interface116, display 114, and/or sensor(s)/system(s) 117 of the marineelectronic device 905 may, for example, provide means for performingoperation 972. Operation 974 may comprise transmitting a control signalfrom the marine electronic device 905 and/or receiving the controlsignal at the unmanned vehicle 920, wherein the control signal includesinstructions for the unmanned vehicle to operate accordingly (such asgathering operation data). The processor 111 and/or communicationinterface 113 of the marine electronic device 905 and/or the processor121 and/or communication interface 123 of the unmanned vehicle 920 may,for example, provide means for performing operation 974.

Operation 976 may comprise causing the operational data to be acquiredby one or more components of the unmanned vehicle. The processor 121,memory 122, communication interface 123, location sensor 128, propulsionsystem 125, and/or other sensor(s)/system(s) 130 may, for example,provide means for performing operation 976. Operation 978 may compriseretrieving (or determining) location data indicative of a currentlocation of the unmanned vehicle 920. The processor 121, memory 122,communication interface 123, location sensor 128, and/or othersensor(s)/system(s) 130 of the unmanned vehicle 920 may, for example,provide means for performing operation 950. Operation 980 may comprisetransmitting the operational data and the location data from theunmanned vehicle 920 and/or receiving the operational data and thelocation data at the marine electronic device 905. The processor 121and/or communication interface 123 of the unmanned vehicle 920 and/orthe processor 111 and/or communication interface 113 of the marineelectronic device 905 may, for example, provide means for performingoperation 980.

Operation 982 may comprise causing display of one or more imagesassociated with the received operational data and/or location data on ascreen of the marine electronic device 905. The processor 111, memory112, communication interface 113, user interface 116, display 114,and/or sensor(s)/system(s) 117 of the marine electronic device 905 may,for example, provide means for performing operation 982.

FIGS. 21-22 illustrate flow control diagrams of a system, method, andcomputer program product according to an example embodiment. It will beunderstood that each block of the flow control diagrams, andcombinations of blocks in the flow control diagrams, may be implementedby various means, such as hardware and/or a computer program productcomprising one or more computer-readable mediums having computerreadable program instructions stored thereon. For example, one or moreof the procedures described herein may be embodied by computer programinstructions of a computer program product. In this regard, the computerprogram product(s) which embody the procedures described herein may bestored by, for example, the memory 112/122 and executed by, for example,the processor 111/121. As will be appreciated, any such computer programproduct may be loaded onto a computer or other programmable apparatus(for example, a marine electronic device 105 and/or an unmanned vehicle120), such that the computer program product including the instructionswhich execute on the computer or other programmable apparatus createsmeans for implementing the functions specified in the flow controldiagrams block(s). Further, the computer program product may compriseone or more non-transitory computer-readable mediums on which thecomputer program instructions may be stored such that the one or morecomputer-readable memories can direct a computer or other programmabledevice (for example, a marine electronic device 105 and/or an unmannedvehicle 120) to cause a series of operations to be performed on thecomputer or other programmable apparatus to produce acomputer-implemented process such that the instructions which execute onthe computer or other programmable apparatus implement the functionsspecified in the flow control diagrams block(s).

Conclusion

Many modifications and other embodiments of the inventions set forthherein will come to mind to one skilled in the art to which theseinventions pertain having the benefit of the teachings presented in theforegoing descriptions and the associated drawings. Therefore, it is tobe understood that the embodiments of the invention are not to belimited to the specific embodiments disclosed and that modifications andother embodiments are intended to be included within the scope of theinvention. Moreover, although the foregoing descriptions and theassociated drawings describe example embodiments in the context ofcertain example combinations of elements and/or functions, it should beappreciated that different combinations of elements and/or functions maybe provided by alternative embodiments without departing from the scopeof the invention. In this regard, for example, different combinations ofelements and/or functions than those explicitly described above are alsocontemplated within the scope of the invention. Although specific termsare employed herein, they are used in a generic and descriptive senseonly and not for purposes of limitation.

1. A marine electronic device of a marine vessel configured for control of an unmanned vehicle, the marine electronic device comprising: a transmitter configured to transmit instructions to the unmanned vehicle; a receiver configured to receive data from the unmanned vehicle; a processor; and a memory including computer program product stored thereon, wherein the computer program product is configured, when executed by the processor, to: receive location data from the unmanned vehicle; determine, based on the received location data, the current location of the unmanned vehicle; determine, based on a planned route or waypoint stored in the memory of the marine electronic device, a desired location for the unmanned vehicle; transmit a control signal to the unmanned vehicle to instruct the unmanned vehicle to travel to the desired location, wherein the control signal causes the unmanned vehicle to travel to the desired location; receive operational data from the unmanned vehicle, wherein the operational data comprises at least sonar data obtained by a sonar system of the unmanned vehicle; and cause display of an image corresponding to the sonar data over a chart or map at the determined location of the unmanned vehicle.
 2. The marine electronic device of claim 1, wherein the computer program product is further configured, when executed by the processor, to: transmit a series of control signals to the unmanned vehicle, wherein each control signal includes instructions to cause the unmanned vehicle to travel to a new desired location so as to cause the unmanned vehicle to autonomously follow the planned route stored in the memory of the marine electronic device.
 3. The marine electronic device of claim 1, wherein the computer program product is further configured, when executed by the processor, to: cause display of a visual representation of the unmanned vehicle on the chart or map relative to the image corresponding to the sonar data.
 4. The marine electronic device of claim 1, wherein the unmanned vehicle is an unmanned underwater vehicle such that a propulsion system of the unmanned vehicle is configured to cause the unmanned underwater vehicle to travel beneath a surface of a body of water of the marine environment.
 5. The marine electronic device of claim 1, wherein the unmanned vehicle is an unmanned aerial vehicle such that the propulsion system is configured to cause the unmanned aerial vehicle to fly around the marine environment.
 6. The marine electronic device of claim 1, wherein the computer program product is further configured, when executed by the processor, to: receive orientation data from the unmanned vehicle; determine the orientation of the unmanned vehicle; and cause display of the image corresponding to the sonar data over the chart or map in an orientation with respect to a visual representation of the unmanned vehicle based on the determined orientation of the unmanned vehicle.
 7. The marine electronic device of claim 1, wherein the operational data includes an indication of an area of coverage of the sonar system, and wherein the computer program product is further configured, when executed by the processor, to: determine the area of coverage of the sonar system; and cause display of a visual representation of the area of coverage over the chart or map.
 8. The marine electronic device of claim 7, wherein the computer program product is further configured, when executed by the processor, to cause display of the image corresponding to the sonar data within the visual representation of the area of coverage.
 9. The marine electronic device of claim 1, wherein the computer program product is further configured, when executed by the processor, to cause display of a plurality of images corresponding to a respective plurality of sets of historical sonar data over the chart or map at respective locations on the chart or map corresponding to when each of the plurality of sets of historical sonar data was captured by the unmanned vehicle.
 10. An unmanned vehicle for a marine environment, wherein the unmanned vehicle is controllable by a marine electronic device of a marine vessel, the unmanned vehicle comprising: a location sensor configured to gather location data corresponding to the unmanned vehicle; a propulsion system configured to propel the unmanned vehicle; at least one operational component configured to gather operational data, wherein the operational component comprises a sonar system configured to collect sonar data from an underwater environment; a transmitter configured to transmit data to the marine electronic device; a receiver configured to receive instructions from the marine electronic device; a processor; and a memory including computer program product stored thereon, wherein the computer program product is configured, when executed by the processor, to: determine location data indicative of a current location of the unmanned vehicle; transmit the location data to the marine electronic device; receive a control signal from the marine electronic device, wherein the control signal includes instructions to travel to a desired location based on a planned route or waypoint stored in memory of the marine electronic device; cause the propulsion system to propel the unmanned vehicle to the desired location; receive operational data from the at least one operational component, wherein the operational data comprises at least the sonar data; and transmit the operational data to the marine electronic device for display of one or more images corresponding to the sonar data over a chart or map at the current location along with a visual representation of the unmanned vehicle at the current location.
 11. The unmanned vehicle of claim 10, wherein the computer program product is further configured, when executed by the processor, to receive a series of control signals from the marine electronic device, wherein each control signal includes instructions to travel to a new desired location so as to cause the unmanned vehicle to autonomously follow the planned route stored in the memory of the marine electronic device.
 12. The unmanned vehicle of claim 10, wherein the computer program product is further configured, when executed by the processor, to cause the propulsion system to land the unmanned vehicle on a surface of a body of water such that at least a portion of the sonar system is submerged so as to gather the sonar data from the underwater environment.
 13. The unmanned vehicle of claim 10, wherein the sonar system is deployable, and wherein the computer program product is further configured, when executed by the processor, to: cause the propulsion system to cause the unmanned vehicle to hover at a hover position above a surface of a body of water; and cause the sonar system to deploy from the hover position to beneath the surface of the body of water such that at least a portion of the sonar system is submerged so as to gather sonar data from the underwater environment.
 14. The unmanned vehicle of claim 10, wherein the unmanned vehicle is an unmanned underwater vehicle such that the propulsion system is configured to cause the unmanned underwater vehicle to travel beneath a surface of a body of water of the marine environment.
 15. The unmanned vehicle of claim 10, wherein the unmanned vehicle is an unmanned aerial vehicle such that the propulsion system is configured to cause the unmanned aerial vehicle to fly around the marine environment.
 16. The unmanned vehicle of claim 10, wherein the unmanned vehicle is an unmanned surface vehicle such that the propulsion system is configured to cause the unmanned surface vehicle to travel along a surface of a body of water of the marine environment.
 17. The unmanned vehicle of claim 10 further comprising an orientation sensor configured to determine an orientation of the unmanned vehicle, wherein the computer program product is further configured, when executed by the processor, to: receive orientation data from the orientation sensor; and transmit the orientation data to the marine electronic device to enable display of the sonar data over the chart or map in an orientation with respect to the visual representation of the unmanned vehicle.
 18. The unmanned vehicle of claim 10, wherein the operational data includes an indication of an area of coverage of the sonar system.
 19. A system for operating an unmanned vehicle in a marine environment, the system comprising: an unmanned vehicle comprising: a location sensor configured to gather location data corresponding to the unmanned vehicle; a propulsion system configured to propel the unmanned vehicle; at least one operational component configured to gather operational data, wherein the operational component comprises a sonar system configured to collect sonar data from an underwater environment; a transmitter configured to transmit data to the marine electronic device; a receiver configured to receive instructions from the marine electronic device; an unmanned vehicle processor; and a memory including unmanned vehicle computer program product stored thereon; and a marine electronic device of a marine vessel configured for control of the unmanned vehicle, the marine electronic device comprising: a transmitter configured to transmit instructions to the unmanned vehicle; a receiver configured to receive data from the unmanned vehicle; a marine electronic device processor; and a memory including marine electronic device computer program product stored thereon; wherein the unmanned vehicle computer program product is configured, when executed by the unmanned vehicle processor, to: determine location data indicative of a current location of the unmanned vehicle; transmit the location data to the marine electronic device; receive operational data from the at least one operational component, wherein the operational data comprises at least sonar data; and transmit the operational data to the marine electronic device; wherein the marine electronic device computer program product is configured, when executed by the marine electronic device processor, to: receive location data from the unmanned vehicle; determine, based on the received location data, the current location of the unmanned vehicle; determine, based on a planned route or waypoint stored in the memory of the marine electronic device, a desired location for the unmanned vehicle; transmit a control signal to the unmanned vehicle to instruct the unmanned vehicle to travel to the desired location, wherein the control signal causes the unmanned vehicle to travel to the desired location; receive operational data from the unmanned vehicle, wherein the operational data comprises at least the sonar data obtained by the unmanned vehicle; and cause display of an image corresponding to the sonar data over a chart or map at the determined location of the unmanned vehicle; and wherein the unmanned vehicle computer program product is further configured, when executed by the unmanned vehicle processor, to: receive a control signal from the marine electronic device, wherein the control signal includes instructions to travel to a desired location based on a planned route or waypoint stored in memory of the marine electronic device; and cause the propulsion system to propel the unmanned vehicle to the desired location.
 20. The system of claim 19, wherein the unmanned vehicle further comprises an orientation sensor configured to determine an orientation of the unmanned vehicle, wherein the unmanned vehicle computer program product is further configured, when executed by the unmanned vehicle processor, to: receive orientation data from the orientation sensor; and transmit the orientation data to the marine electronic device; and wherein the marine electronic device computer program product is further configured, when executed by the marine electronic device processor, to: receive the orientation data from the unmanned vehicle; determine the orientation of the unmanned vehicle; and cause display of the image corresponding to the sonar data over the chart or map in an orientation with respect to a visual representation of the unmanned vehicle based on the determined orientation of the unmanned vehicle. 